Jul 19 14:46:57.769667 ip-10-0-142-137 systemd[1]: Starting Kubernetes Kubelet...
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.146097    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150530    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150544    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150548    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150551    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150562    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150565    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150568    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150571    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:58.198285 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150574    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150576    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150579    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150581    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150584    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150586    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150589    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150592    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150595    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150597    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150600    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150602    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150605    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150608    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150610    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150613    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150615    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150618    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150621    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:58.200575 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150624    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150626    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150629    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150632    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150635    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150638    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150640    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150643    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150645    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150647    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150650    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150653    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150656    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150658    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150661    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150663    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150666    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150671    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150675    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:58.202799 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150678    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150681    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150683    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150686    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150689    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150691    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150695    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150699    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150703    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150706    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150710    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150712    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150715    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150718    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150720    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150723    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150725    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150728    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150731    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150733    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:58.204895 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150736    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150738    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150741    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150744    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150747    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150750    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150752    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150755    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150757    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150760    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150762    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150765    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150768    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150770    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150774    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150777    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150779    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150782    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150786    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:58.205832 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150790    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150793    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150796    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150799    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150802    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150805    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150808    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150810    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150813    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150816    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150818    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150821    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150823    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.150825    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151528    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151535    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151538    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151541    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151544    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151547    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:58.207496 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151550    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151553    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151556    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151558    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151561    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151564    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151566    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151569    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151572    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151574    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151577    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151579    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151582    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151584    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151587    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151590    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151593    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151595    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151598    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151601    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:58.208963 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151604    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151607    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151609    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151612    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151614    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151617    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151620    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151622    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151625    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151627    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151630    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151633    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151636    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151638    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151641    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151644    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151646    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151649    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151652    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:58.209838 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151654    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151656    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151659    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151661    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151664    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151666    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151669    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151671    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151674    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151676    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151681    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151685    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151688    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151691    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151694    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151696    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151699    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151701    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151704    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151706    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:58.210729 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151709    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151711    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151714    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151717    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151720    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151723    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151726    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151728    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151733    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151737    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151740    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151744    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151747    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151753    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151756    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151759    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151762    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151764    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151766    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:58.211255 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151770    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151773    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151776    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151778    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151790    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151793    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151796    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151798    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151801    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151804    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151806    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151809    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151812    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151814    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.151817    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152170    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152177    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152181    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152185    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152188    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:58.211831 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152191    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152194    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152196    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152199    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152201    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152204    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152208    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152211    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152214    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152217    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152220    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152222    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152226    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152230    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152233    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152236    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152239    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152242    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152245    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:58.212721 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152248    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152251    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152254    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152256    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152259    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152261    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152264    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152266    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152269    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152273    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152275    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152278    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152281    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152283    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152286    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152289    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152291    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152294    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152296    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152299    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:58.213397 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152301    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152304    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152306    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152310    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152312    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152315    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152317    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152319    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152322    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152325    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152328    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152331    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152335    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152337    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152340    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152343    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152345    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152348    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152350    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152353    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:58.214075 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152355    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152358    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152360    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152364    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152367    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152370    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152373    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152375    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152378    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152380    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152383    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152386    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152389    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152392    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152394    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152396    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152410    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152430    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152435    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152439    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:58.214712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152442    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152445    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152448    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152451    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152453    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152456    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152459    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152461    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152464    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152466    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152471    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152474    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152477    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152479    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.152482    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152553    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152560    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152566    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152571    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152575    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 19 14:46:58.215383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152578    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152585    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152589    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152593    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152596    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152599    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152603    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152606    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152609    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152612    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152615    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152618    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152621    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152625    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152628    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152631    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152634    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152638    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152641    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152645    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152648    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152652    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152657    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152660    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 19 14:46:58.216234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152663    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152666    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152669    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152675    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152678    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152681    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152684    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152687    2577 flags.go:64] FLAG: --enable-server="true"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152690    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152695    2577 flags.go:64] FLAG: --event-burst="100"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152698    2577 flags.go:64] FLAG: --event-qps="50"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152701    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152704    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152707    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152710    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152713    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152716    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152719    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152722    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152725    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152728    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152731    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152734    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152737    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152740    2577 flags.go:64] FLAG: --feature-gates=""
Jul 19 14:46:58.216875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152744    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152747    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152750    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152753    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152756    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152761    2577 flags.go:64] FLAG: --help="false"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152764    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-137.ec2.internal"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152767    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152770    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152772    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152777    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152781    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152783    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152786    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152789    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152793    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152796    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152799    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152802    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152805    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152808    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152811    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152813    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152816    2577 flags.go:64] FLAG: --lock-file=""
Jul 19 14:46:58.217726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152819    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152822    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152825    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152830    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152833    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152836    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152839    2577 flags.go:64] FLAG: --logging-format="text"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152842    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152846    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152848    2577 flags.go:64] FLAG: --manifest-url=""
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152851    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152855    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152858    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152864    2577 flags.go:64] FLAG: --max-pods="110"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152867    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152869    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152872    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152875    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152879    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152882    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152885    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152892    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152895    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152898    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 19 14:46:58.220939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152903    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152906    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152911    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152914    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152917    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152920    2577 flags.go:64] FLAG: --port="10250"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152923    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152926    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-095a247c61accb686"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152929    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152932    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152935    2577 flags.go:64] FLAG: --register-node="true"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152938    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152941    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152944    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152947    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152950    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152953    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152956    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152959    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152962    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152965    2577 flags.go:64] FLAG: --runonce="false"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152969    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152972    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152975    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152978    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 19 14:46:58.221791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152980    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152983    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152988    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152991    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152994    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152997    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.152999    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153002    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153005    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153008    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153011    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153016    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153019    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153022    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153025    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153028    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153031    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153034    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153036    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153039    2577 flags.go:64] FLAG: --v="2"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153043    2577 flags.go:64] FLAG: --version="false"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153047    2577 flags.go:64] FLAG: --vmodule=""
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153050    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.153054    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153193    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:58.305110 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153197    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:58.228763 ip-10-0-142-137 systemd[1]: Started Kubernetes Kubelet.
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153201    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153204    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153208    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153212    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153215    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153217    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153220    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153223    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153228    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153231    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153234    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153236    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153239    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153242    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153244    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153248    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153250    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153253    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153256    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:58.306039 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153259    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153263    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153266    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153269    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153272    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153276    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153279    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153283    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153285    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153289    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153291    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153294    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153297    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153300    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153303    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153307    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153310    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153313    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153315    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:58.306827 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153318    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153321    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153324    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153327    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153330    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153332    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153335    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153338    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153340    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153343    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153346    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153348    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153351    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153353    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153356    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153359    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153361    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153364    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153367    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153369    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:58.307878 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153372    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153374    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153377    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153379    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153382    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153385    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153388    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153390    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153394    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153397    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153399    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153402    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153404    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153407    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153411    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153426    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153431    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153435    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153439    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:58.308712 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153443    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153448    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153452    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153455    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153458    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153461    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153464    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153467    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153469    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153472    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153475    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153478    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153480    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153483    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153485    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153488    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153490    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153493    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153496    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:58.309318 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.153498    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.154088    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.161980    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.161995    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.162770    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.162778    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162835    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162841    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162844    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162847    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162850    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162852    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162855    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162858    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162861    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:58.309936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162864    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162867    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162870    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162873    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162876    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162878    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162881    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162883    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162887    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162891    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162894    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162897    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162900    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162903    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162906    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162909    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162911    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162914    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162916    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:58.310786 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162919    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162922    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162927    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162930    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162933    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162936    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162940    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162943    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162945    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162948    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162951    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162953    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162956    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162959    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162961    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162964    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162967    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162969    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162973    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162976    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:58.311311 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162979    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162981    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162984    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162986    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162989    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162991    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162994    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162996    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.162999    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163002    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163005    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163007    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163010    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163012    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163016    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163019    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163021    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163024    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163028    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:58.311951 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163030    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163033    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163036    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163038    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163042    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163045    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163048    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163050    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163053    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163056    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163058    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163061    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163064    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163067    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163069    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163072    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163074    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163077    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163079    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163082    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:58.312600 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163085    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163087    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163090    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163092    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163095    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163097    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163101    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163106    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163109    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163112    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163114    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163119    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.163125    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163233    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:58.313168 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163238    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163240    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163243    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163246    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163249    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163252    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163255    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163258    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163261    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163264    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163267    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163271    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163273    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163276    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163279    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163281    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163284    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163287    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163289    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163292    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:58.313584 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163294    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163297    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163301    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163305    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163307    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163312    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163315    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163317    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163321    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163323    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163328    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163330    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163333    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163336    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163339    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163341    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163344    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163346    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163349    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:58.314144 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163351    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163354    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163357    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163360    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163362    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163367    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163370    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163373    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163375    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163377    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163380    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163383    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163385    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163388    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163390    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163393    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163396    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163398    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163402    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163404    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:58.314673 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163407    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163409    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163412    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163434    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163437    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163440    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163443    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163446    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163448    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163451    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163454    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163457    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163459    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163462    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163465    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163467    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163470    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163472    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163476    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:58.315429 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163479    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163482    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163484    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163487    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163489    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163492    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163494    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163497    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163500    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163502    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163506    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163511    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163514    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163517    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163519    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163522    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163527    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163530    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163533    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:58.316319 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:46:58.163535    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.163541    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.163634    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.166290    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.167583    2577 server.go:1019] "Starting client certificate rotation"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.167679    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.167719    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.170433    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.178514    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.187387    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.190057    2577 log.go:25] "Validated CRI v1 image API"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.190990    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 19 14:46:58.317285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.193315    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:46:58.317923 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.201241    2577 fs.go:135] Filesystem UUIDs: map[2255f9b9-ddba-4255-8ea4-8d29a1be1c62:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8d5d095c-e12a-49aa-b412-21eb4e2dc593:/dev/nvme0n1p4]
Jul 19 14:46:58.317923 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.201256    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.207065    2577 manager.go:216] Machine: {Timestamp:2026-07-19 14:46:58.205827758 +0000 UTC m=+0.321914958 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091411 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24884b590102882e7d48ae3ca51b4f SystemUUID:ec24884b-5901-0288-2e7d-48ae3ca51b4f BootID:d63fb20b-48c3-486a-bff0-e89b38b42b31 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ac:12:3a:9e:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ac:12:3a:9e:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:b4:21:ca:46:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.207164    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.207231    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.209979    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.210007    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-137.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.210729    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.210739    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.210746    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.211514    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.212673    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215286    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nv4dw"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215410    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215554    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215587    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215600    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215617    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 19 14:46:58.318704 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215629    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.215640    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.216000    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.216020    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.218219    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.220232    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.220263    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221134    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221151    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221160    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221169    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221181    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221190    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221204    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221212    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221222    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221230    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221240    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221247    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221985    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.221993    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.226836    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nv4dw"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.228193    2577 server.go:1268] "Started kubelet"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.228543    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.230492    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.231901    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.231946    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.234176    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.234466    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.236641    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.236874    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.237213    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.237801    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.237813    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.237901    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.237948    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.237954    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.238376    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-137.ec2.internal\" not found"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.238519    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.238793    2577 factory.go:55] Registering systemd factory
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.238808    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.250764    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.250816    2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-137.ec2.internal" not found
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.252381    2577 factory.go:153] Registering CRI-O factory
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.252397    2577 factory.go:223] Registration of the crio container factory successfully
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.252475    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.252497    2577 factory.go:103] Registering Raw factory
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.252512    2577 manager.go:1196] Started watching for new ooms in manager
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.252917    2577 manager.go:349] Starting recovery of all containers
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.253002    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-137.ec2.internal\" not found" node="ip-10-0-142-137.ec2.internal"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.263837    2577 manager.go:354] Recovery completed
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.267596    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:58.319673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279061    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279097    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279115    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279645    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279661    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279665    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.279705    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.280537    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.280562    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.280573    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.281073    2577 policy_none.go:49] "None policy: Start"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.281087    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.281097    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.282136    2577 policy_none.go:47] "Start"
Jul 19 14:46:58.322192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.282609    2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-137.ec2.internal" not found
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.341139    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-137.ec2.internal\" not found"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.343361    2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-137.ec2.internal" not found
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345522    2577 manager.go:337] "Starting Device Plugin manager"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.345547    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345557    2577 server.go:87] "Starting device plugin registration server"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345789    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345801    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345881    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345953    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.345964    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.346466    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.346513    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-137.ec2.internal\" not found"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.440548    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.441645    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.441682    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.441700    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:58.441737    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.443897    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.446724    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.447488    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.447514    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.447533    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.447557    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-137.ec2.internal"
Jul 19 14:46:58.474269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.456120    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.542126    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"]
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.544000    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.544868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.550741    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.564695    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:58.566988    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.667193    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.670086    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.672055    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.685631    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:59.685688    2577 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-137.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.685720    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:46:59.732638 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:46:59.685755    2577 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.734104 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.733406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49b4d8c3647ee516d9bd0bef4be2f438-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" (UID: \"49b4d8c3647ee516d9bd0bef4be2f438\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.734104 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.733438    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49b4d8c3647ee516d9bd0bef4be2f438-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" (UID: \"49b4d8c3647ee516d9bd0bef4be2f438\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.734104 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.733460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e72960bc383aaed664cf3e1030c9130-config\") pod \"kube-apiserver-proxy-ip-10-0-142-137.ec2.internal\" (UID: \"6e72960bc383aaed664cf3e1030c9130\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.834412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.834225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49b4d8c3647ee516d9bd0bef4be2f438-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" (UID: \"49b4d8c3647ee516d9bd0bef4be2f438\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.834412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.834247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49b4d8c3647ee516d9bd0bef4be2f438-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" (UID: \"49b4d8c3647ee516d9bd0bef4be2f438\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.834412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.834263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e72960bc383aaed664cf3e1030c9130-config\") pod \"kube-apiserver-proxy-ip-10-0-142-137.ec2.internal\" (UID: \"6e72960bc383aaed664cf3e1030c9130\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.834412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.834299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e72960bc383aaed664cf3e1030c9130-config\") pod \"kube-apiserver-proxy-ip-10-0-142-137.ec2.internal\" (UID: \"6e72960bc383aaed664cf3e1030c9130\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.834412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.834330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49b4d8c3647ee516d9bd0bef4be2f438-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" (UID: \"49b4d8c3647ee516d9bd0bef4be2f438\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.834412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.834337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49b4d8c3647ee516d9bd0bef4be2f438-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal\" (UID: \"49b4d8c3647ee516d9bd0bef4be2f438\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:46:59.988334 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.988215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal"
Jul 19 14:47:00.108839 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:46:59.990242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal"
Jul 19 14:47:00.261596 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.258344    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 19 14:47:00.261596 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.258459    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:47:00.261596 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.258489    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:47:00.261596 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.258526    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.307483    2577 apiserver.go:52] "Watching apiserver"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.315433    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.315817    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-pdcdf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal","openshift-multus/multus-additional-cni-plugins-zcsxf","openshift-multus/multus-s4nr8","openshift-network-diagnostics/network-check-target-svvdc","openshift-network-operator/iptables-alerter-r7jxc","openshift-ovn-kubernetes/ovnkube-node-bjgvh","kube-system/global-pull-secret-syncer-kpllx","kube-system/konnectivity-agent-2vssd","kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n","openshift-cluster-node-tuning-operator/tuned-2xxhx","openshift-dns/node-resolver-wxdbs","openshift-image-registry/node-ca-7n5n2"]
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.317349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.317406    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.318545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.319162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.320905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.320962    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321190    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-chv22\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321382    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321667    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321799    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321938    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321932    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-18 14:41:59 +0000 UTC" deadline="2028-03-14 20:26:32.705203331 +0000 UTC"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.321974    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14501h39m32.383233054s"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.322039    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.322473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.322485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.322636    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8z8f8\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.322865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.324055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.324102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.324424    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.324692    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.324728    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5pjq4\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.324983    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325010    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.366561 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325060    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bm5qs\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325271    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325776    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325814    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.325828    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.326334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.327622    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.328403    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.328575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-55krb\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.328707    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.328792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.329058    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8tv67\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.329364    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.329450    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.329641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.330077    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.331100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.331568    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.331652    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.331907    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2whms\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.332544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.333609    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.333758    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.333990    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-248vh\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.334468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.334822    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.335020    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.335134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hh2kr\""
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-kubelet-config\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-host\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-daemon-config\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.368847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-device-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-socket-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-registration-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa88057d-da49-49e6-ac70-a17af00c1060-hosts-file\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fcdb84c-934b-466b-874b-d85c7a45b15c-iptables-alerter-script\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-systemd\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339602    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339621    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24aa3357-ff95-449c-b1b5-fda0fac5df4a-agent-certs\") pod \"konnectivity-agent-2vssd\" (UID: \"24aa3357-ff95-449c-b1b5-fda0fac5df4a\") " pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-kubernetes\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysctl-d\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6chp\" (UniqueName: \"kubernetes.io/projected/d09e1cc9-a996-410f-aee1-833b56eeda4c-kube-api-access-p6chp\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6vnb\" (UniqueName: \"kubernetes.io/projected/89704d43-d416-4b1b-853c-52ecefa8b7a1-kube-api-access-q6vnb\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-cni-netd\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.371197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-socket-dir-parent\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-conf-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-multus-certs\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-slash\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-run-netns\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa88057d-da49-49e6-ac70-a17af00c1060-tmp-dir\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-system-cni-dir\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.339982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-cni-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-cni-bin\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-etc-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-systemd\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysconfig\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-sys\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.372232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09e1cc9-a996-410f-aee1-833b56eeda4c-tmp\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-kubelet\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fcdb84c-934b-466b-874b-d85c7a45b15c-host-slash\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-cni-bin\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-tuned\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4d95ccc-1c36-4ac7-b655-937995dd14ac-cni-binary-copy\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6t29\" (UniqueName: \"kubernetes.io/projected/e4d95ccc-1c36-4ac7-b655-937995dd14ac-kube-api-access-w6t29\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-kubelet\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-node-log\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-log-socket\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysctl-conf\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-run\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-var-lib-kubelet\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-etc-kubernetes\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.374783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-sys-fs\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-ovnkube-script-lib\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-dbus\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-os-release\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-lib-modules\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-os-release\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-cni-multus\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmsgm\" (UniqueName: \"kubernetes.io/projected/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-kube-api-access-bmsgm\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-ovnkube-config\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24aa3357-ff95-449c-b1b5-fda0fac5df4a-konnectivity-ca\") pod \"konnectivity-agent-2vssd\" (UID: \"24aa3357-ff95-449c-b1b5-fda0fac5df4a\") " pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-modprobe-d\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-hostroot\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-ovn\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-run-ovn-kubernetes\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.375380 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cnibin\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfhlv\" (UniqueName: \"kubernetes.io/projected/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-kube-api-access-jfhlv\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-system-cni-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-cnibin\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/decd7f26-f653-4505-a3d1-f8a4805c792a-ovn-node-metrics-cert\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdh9\" (UniqueName: \"kubernetes.io/projected/decd7f26-f653-4505-a3d1-f8a4805c792a-kube-api-access-ftdh9\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-k8s-cni-cncf-io\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-netns\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5qjh\" (UniqueName: \"kubernetes.io/projected/fa88057d-da49-49e6-ac70-a17af00c1060-kube-api-access-b5qjh\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfh87\" (UniqueName: \"kubernetes.io/projected/4fcdb84c-934b-466b-874b-d85c7a45b15c-kube-api-access-zfh87\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-systemd-units\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.340995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-var-lib-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.341016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-env-overrides\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.341307    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.359189    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9k99r"
Jul 19 14:47:00.377937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.367297    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9k99r"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.430455    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-socket-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-registration-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa88057d-da49-49e6-ac70-a17af00c1060-hosts-file\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fcdb84c-934b-466b-874b-d85c7a45b15c-iptables-alerter-script\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-systemd\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24aa3357-ff95-449c-b1b5-fda0fac5df4a-agent-certs\") pod \"konnectivity-agent-2vssd\" (UID: \"24aa3357-ff95-449c-b1b5-fda0fac5df4a\") " pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfkg9\" (UniqueName: \"kubernetes.io/projected/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-kube-api-access-lfkg9\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-kubernetes\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysctl-d\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6chp\" (UniqueName: \"kubernetes.io/projected/d09e1cc9-a996-410f-aee1-833b56eeda4c-kube-api-access-p6chp\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6vnb\" (UniqueName: \"kubernetes.io/projected/89704d43-d416-4b1b-853c-52ecefa8b7a1-kube-api-access-q6vnb\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:00.475074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-cni-netd\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-socket-dir-parent\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-conf-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-multus-certs\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-slash\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-run-netns\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa88057d-da49-49e6-ac70-a17af00c1060-tmp-dir\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-system-cni-dir\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-cni-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-cni-bin\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-etc-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-systemd\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysconfig\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-sys\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.477332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09e1cc9-a996-410f-aee1-833b56eeda4c-tmp\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-kubelet\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fcdb84c-934b-466b-874b-d85c7a45b15c-host-slash\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-cni-bin\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-serviceca\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.441992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-tuned\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4d95ccc-1c36-4ac7-b655-937995dd14ac-cni-binary-copy\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6t29\" (UniqueName: \"kubernetes.io/projected/e4d95ccc-1c36-4ac7-b655-937995dd14ac-kube-api-access-w6t29\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-kubelet\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-node-log\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-log-socket\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysctl-conf\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-run\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-var-lib-kubelet\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-etc-kubernetes\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.478067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-sys-fs\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-ovnkube-script-lib\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-dbus\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-os-release\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-lib-modules\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-os-release\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-cni-multus\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmsgm\" (UniqueName: \"kubernetes.io/projected/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-kube-api-access-bmsgm\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-ovnkube-config\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24aa3357-ff95-449c-b1b5-fda0fac5df4a-konnectivity-ca\") pod \"konnectivity-agent-2vssd\" (UID: \"24aa3357-ff95-449c-b1b5-fda0fac5df4a\") " pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-modprobe-d\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-hostroot\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-ovn\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-run-ovn-kubernetes\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-host\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.479092 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cnibin\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhlv\" (UniqueName: \"kubernetes.io/projected/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-kube-api-access-jfhlv\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-system-cni-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-cnibin\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/decd7f26-f653-4505-a3d1-f8a4805c792a-ovn-node-metrics-cert\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdh9\" (UniqueName: \"kubernetes.io/projected/decd7f26-f653-4505-a3d1-f8a4805c792a-kube-api-access-ftdh9\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-k8s-cni-cncf-io\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-netns\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5qjh\" (UniqueName: \"kubernetes.io/projected/fa88057d-da49-49e6-ac70-a17af00c1060-kube-api-access-b5qjh\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfh87\" (UniqueName: \"kubernetes.io/projected/4fcdb84c-934b-466b-874b-d85c7a45b15c-kube-api-access-zfh87\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-systemd-units\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-var-lib-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-env-overrides\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-kubelet-config\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-host\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-daemon-config\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-device-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.479753 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.442924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-device-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.443014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-socket-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.443051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-registration-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.443082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa88057d-da49-49e6-ac70-a17af00c1060-hosts-file\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.443477    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fcdb84c-934b-466b-874b-d85c7a45b15c-iptables-alerter-script\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.443558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-systemd\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.443590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.443654    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.443698    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:47:00.943683765 +0000 UTC m=+1.968692818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.444022    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.446932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24aa3357-ff95-449c-b1b5-fda0fac5df4a-agent-certs\") pod \"konnectivity-agent-2vssd\" (UID: \"24aa3357-ff95-449c-b1b5-fda0fac5df4a\") " pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-kubernetes\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysctl-d\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-cni-netd\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-socket-dir-parent\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-conf-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-multus-certs\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.480627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-slash\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.447965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-run-netns\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448256    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa88057d-da49-49e6-ac70-a17af00c1060-tmp-dir\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-system-cni-dir\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-cni-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-cni-bin\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-etc-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-systemd\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysconfig\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.448987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-sys\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.450721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d09e1cc9-a996-410f-aee1-833b56eeda4c-tmp\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.450768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-kubelet\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.450796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fcdb84c-934b-466b-874b-d85c7a45b15c-host-slash\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.450826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-cni-bin\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.452405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-tuned\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.481587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.452864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4d95ccc-1c36-4ac7-b655-937995dd14ac-cni-binary-copy\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.453104    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.453149    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:00.953134464 +0000 UTC m=+1.978143520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-kubelet\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-node-log\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-log-socket\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-sysctl-conf\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-run\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-var-lib-kubelet\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453849    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-etc-kubernetes\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.453968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-sys-fs\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.454472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-ovnkube-script-lib\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.454561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-dbus\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.454621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-os-release\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.454722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-lib-modules\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.454777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-os-release\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484094 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.454818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-var-lib-cni-multus\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.455317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-ovnkube-config\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.455658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24aa3357-ff95-449c-b1b5-fda0fac5df4a-konnectivity-ca\") pod \"konnectivity-agent-2vssd\" (UID: \"24aa3357-ff95-449c-b1b5-fda0fac5df4a\") " pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.455730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-etc-modprobe-d\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-hostroot\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-run-ovn\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-host-run-ovn-kubernetes\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-cnibin\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-system-cni-dir\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-cnibin\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-netns\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-kubelet-config\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-systemd-units\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/decd7f26-f653-4505-a3d1-f8a4805c792a-var-lib-openvswitch\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.456907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4d95ccc-1c36-4ac7-b655-937995dd14ac-host-run-k8s-cni-cncf-io\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.457306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09e1cc9-a996-410f-aee1-833b56eeda4c-host\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.484852 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.458161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4d95ccc-1c36-4ac7-b655-937995dd14ac-multus-daemon-config\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.458876    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.458881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/decd7f26-f653-4505-a3d1-f8a4805c792a-ovn-node-metrics-cert\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.458907    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.458922    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.459104    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:00.959087393 +0000 UTC m=+1.984096467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.459109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/decd7f26-f653-4505-a3d1-f8a4805c792a-env-overrides\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.460878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6t29\" (UniqueName: \"kubernetes.io/projected/e4d95ccc-1c36-4ac7-b655-937995dd14ac-kube-api-access-w6t29\") pod \"multus-s4nr8\" (UID: \"e4d95ccc-1c36-4ac7-b655-937995dd14ac\") " pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.461154    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6chp\" (UniqueName: \"kubernetes.io/projected/d09e1cc9-a996-410f-aee1-833b56eeda4c-kube-api-access-p6chp\") pod \"tuned-2xxhx\" (UID: \"d09e1cc9-a996-410f-aee1-833b56eeda4c\") " pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.461712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6vnb\" (UniqueName: \"kubernetes.io/projected/89704d43-d416-4b1b-853c-52ecefa8b7a1-kube-api-access-q6vnb\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.463147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmsgm\" (UniqueName: \"kubernetes.io/projected/6c6c3cbb-8e2b-47a5-9118-73680d9a11ab-kube-api-access-bmsgm\") pod \"aws-ebs-csi-driver-node-wgd4n\" (UID: \"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.464221    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhlv\" (UniqueName: \"kubernetes.io/projected/da3846eb-8ab6-4170-bba9-5f7c0886b7c4-kube-api-access-jfhlv\") pod \"multus-additional-cni-plugins-zcsxf\" (UID: \"da3846eb-8ab6-4170-bba9-5f7c0886b7c4\") " pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.465602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfh87\" (UniqueName: \"kubernetes.io/projected/4fcdb84c-934b-466b-874b-d85c7a45b15c-kube-api-access-zfh87\") pod \"iptables-alerter-r7jxc\" (UID: \"4fcdb84c-934b-466b-874b-d85c7a45b15c\") " pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.466051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5qjh\" (UniqueName: \"kubernetes.io/projected/fa88057d-da49-49e6-ac70-a17af00c1060-kube-api-access-b5qjh\") pod \"node-resolver-wxdbs\" (UID: \"fa88057d-da49-49e6-ac70-a17af00c1060\") " pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.485707 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.466198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdh9\" (UniqueName: \"kubernetes.io/projected/decd7f26-f653-4505-a3d1-f8a4805c792a-kube-api-access-ftdh9\") pod \"ovnkube-node-bjgvh\" (UID: \"decd7f26-f653-4505-a3d1-f8a4805c792a\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.508454    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2xxhx"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.543900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfkg9\" (UniqueName: \"kubernetes.io/projected/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-kube-api-access-lfkg9\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.543953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-serviceca\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.544024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-host\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.544151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-host\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.544808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-serviceca\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.553459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wxdbs"
Jul 19 14:47:00.574912 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.564002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfkg9\" (UniqueName: \"kubernetes.io/projected/9af30ba0-f27d-45f2-91d7-bedb8f428ad7-kube-api-access-lfkg9\") pod \"node-ca-7n5n2\" (UID: \"9af30ba0-f27d-45f2-91d7-bedb8f428ad7\") " pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.577000    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b4d8c3647ee516d9bd0bef4be2f438.slice/crio-48fbe487dfa7143576c61d18f4bd224cda57b53164147621aecf15a3f4d71966 WatchSource:0}: Error finding container 48fbe487dfa7143576c61d18f4bd224cda57b53164147621aecf15a3f4d71966: Status 404 returned error can't find the container with id 48fbe487dfa7143576c61d18f4bd224cda57b53164147621aecf15a3f4d71966
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.578009    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa88057d_da49_49e6_ac70_a17af00c1060.slice/crio-7363351c36cb85336ffe6ab27792a08392ba6e55f97c71047dfedc05cce88bad WatchSource:0}: Error finding container 7363351c36cb85336ffe6ab27792a08392ba6e55f97c71047dfedc05cce88bad: Status 404 returned error can't find the container with id 7363351c36cb85336ffe6ab27792a08392ba6e55f97c71047dfedc05cce88bad
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.579085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e72960bc383aaed664cf3e1030c9130.slice/crio-48ca36427407616b93c94d7e008083a92e55b8c7f6664f158017c0ee64ed622d WatchSource:0}: Error finding container 48ca36427407616b93c94d7e008083a92e55b8c7f6664f158017c0ee64ed622d: Status 404 returned error can't find the container with id 48ca36427407616b93c94d7e008083a92e55b8c7f6664f158017c0ee64ed622d
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.584229    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.671369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zcsxf"
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.680064    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3846eb_8ab6_4170_bba9_5f7c0886b7c4.slice/crio-1ecf2979a0052795f4e4356421195f85489984e2c9d907160a431fca231c2870 WatchSource:0}: Error finding container 1ecf2979a0052795f4e4356421195f85489984e2c9d907160a431fca231c2870: Status 404 returned error can't find the container with id 1ecf2979a0052795f4e4356421195f85489984e2c9d907160a431fca231c2870
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.689637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s4nr8"
Jul 19 14:47:00.703936 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.697073    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d95ccc_1c36_4ac7_b655_937995dd14ac.slice/crio-30a8aff979794554ebf31d12590b31d92fa76c1bf35c54913cfbf7819a2ea9df WatchSource:0}: Error finding container 30a8aff979794554ebf31d12590b31d92fa76c1bf35c54913cfbf7819a2ea9df: Status 404 returned error can't find the container with id 30a8aff979794554ebf31d12590b31d92fa76c1bf35c54913cfbf7819a2ea9df
Jul 19 14:47:00.709413 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.709396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7jxc"
Jul 19 14:47:00.715155 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.715128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:00.719626 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.719594    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fcdb84c_934b_466b_874b_d85c7a45b15c.slice/crio-133a2da35fd6d737833836527af3b3cba2755cca8c853a251bf035083a0e1743 WatchSource:0}: Error finding container 133a2da35fd6d737833836527af3b3cba2755cca8c853a251bf035083a0e1743: Status 404 returned error can't find the container with id 133a2da35fd6d737833836527af3b3cba2755cca8c853a251bf035083a0e1743
Jul 19 14:47:00.724366 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.724348    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecd7f26_f653_4505_a3d1_f8a4805c792a.slice/crio-632323eafe2dbde1740a9ed39fd7eeb63adfed444aa9257815d3485586917ae5 WatchSource:0}: Error finding container 632323eafe2dbde1740a9ed39fd7eeb63adfed444aa9257815d3485586917ae5: Status 404 returned error can't find the container with id 632323eafe2dbde1740a9ed39fd7eeb63adfed444aa9257815d3485586917ae5
Jul 19 14:47:00.733840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.733823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:00.739651 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.739636    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n"
Jul 19 14:47:00.741420 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.741401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24aa3357_ff95_449c_b1b5_fda0fac5df4a.slice/crio-9d7f412051bfecbbcd9d24d5a8990e731ea944a9ffa6483f7e52b591d2c96ddc WatchSource:0}: Error finding container 9d7f412051bfecbbcd9d24d5a8990e731ea944a9ffa6483f7e52b591d2c96ddc: Status 404 returned error can't find the container with id 9d7f412051bfecbbcd9d24d5a8990e731ea944a9ffa6483f7e52b591d2c96ddc
Jul 19 14:47:00.757267 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.747165    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6c3cbb_8e2b_47a5_9118_73680d9a11ab.slice/crio-b3e9a6027b3abdf252bbd83793345ded0adfb752ec804f2fbdb7d032d3b3c31d WatchSource:0}: Error finding container b3e9a6027b3abdf252bbd83793345ded0adfb752ec804f2fbdb7d032d3b3c31d: Status 404 returned error can't find the container with id b3e9a6027b3abdf252bbd83793345ded0adfb752ec804f2fbdb7d032d3b3c31d
Jul 19 14:47:00.860126 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.860100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7n5n2"
Jul 19 14:47:00.868702 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:00.868678    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af30ba0_f27d_45f2_91d7_bedb8f428ad7.slice/crio-00488f536304b011c06340ab523500d32b7308fe36c24d9c921c58aadee20079 WatchSource:0}: Error finding container 00488f536304b011c06340ab523500d32b7308fe36c24d9c921c58aadee20079: Status 404 returned error can't find the container with id 00488f536304b011c06340ab523500d32b7308fe36c24d9c921c58aadee20079
Jul 19 14:47:00.946925 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:00.946901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:00.947412 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.947385    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:00.947546 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:00.947466    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:47:01.947438768 +0000 UTC m=+2.972447825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:01.020518 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:01.020102    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09e1cc9_a996_410f_aee1_833b56eeda4c.slice/crio-a1fac639026dba1a6e76ebba1f0161c1c34695362004f267bc1b91ae8b97905c WatchSource:0}: Error finding container a1fac639026dba1a6e76ebba1f0161c1c34695362004f267bc1b91ae8b97905c: Status 404 returned error can't find the container with id a1fac639026dba1a6e76ebba1f0161c1c34695362004f267bc1b91ae8b97905c
Jul 19 14:47:01.047755 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.047726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:01.047888 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.047784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:01.047957 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.047918    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:01.048006 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.047978    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:02.047958572 +0000 UTC m=+3.072967625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:01.048467 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.048439    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:01.048467 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.048467    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:01.048636 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.048480    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:01.048636 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.048541    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:02.048525221 +0000 UTC m=+3.073534278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:01.370323 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.370135    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:42:00 +0000 UTC" deadline="2028-04-11 08:50:48.75970583 +0000 UTC"
Jul 19 14:47:01.370323 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.370194    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15162h3m47.389515939s"
Jul 19 14:47:01.404415 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.404119    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:47:01.536151 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.536121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:01.536329 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.536252    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:01.536664 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.536642    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:01.536759 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.536738    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:01.590257 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.590133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2xxhx" event={"ID":"d09e1cc9-a996-410f-aee1-833b56eeda4c","Type":"ContainerStarted","Data":"a1fac639026dba1a6e76ebba1f0161c1c34695362004f267bc1b91ae8b97905c"}
Jul 19 14:47:01.609245 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.609210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n" event={"ID":"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab","Type":"ContainerStarted","Data":"b3e9a6027b3abdf252bbd83793345ded0adfb752ec804f2fbdb7d032d3b3c31d"}
Jul 19 14:47:01.644331 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.644094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"632323eafe2dbde1740a9ed39fd7eeb63adfed444aa9257815d3485586917ae5"}
Jul 19 14:47:01.669080 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.669041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7jxc" event={"ID":"4fcdb84c-934b-466b-874b-d85c7a45b15c","Type":"ContainerStarted","Data":"133a2da35fd6d737833836527af3b3cba2755cca8c853a251bf035083a0e1743"}
Jul 19 14:47:01.697962 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.697813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s4nr8" event={"ID":"e4d95ccc-1c36-4ac7-b655-937995dd14ac","Type":"ContainerStarted","Data":"30a8aff979794554ebf31d12590b31d92fa76c1bf35c54913cfbf7819a2ea9df"}
Jul 19 14:47:01.722892 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.722851    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:47:01.724913 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.724882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerStarted","Data":"1ecf2979a0052795f4e4356421195f85489984e2c9d907160a431fca231c2870"}
Jul 19 14:47:01.738359 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.738327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wxdbs" event={"ID":"fa88057d-da49-49e6-ac70-a17af00c1060","Type":"ContainerStarted","Data":"7363351c36cb85336ffe6ab27792a08392ba6e55f97c71047dfedc05cce88bad"}
Jul 19 14:47:01.744119 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.744091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal" event={"ID":"49b4d8c3647ee516d9bd0bef4be2f438","Type":"ContainerStarted","Data":"48fbe487dfa7143576c61d18f4bd224cda57b53164147621aecf15a3f4d71966"}
Jul 19 14:47:01.752924 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.752898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7n5n2" event={"ID":"9af30ba0-f27d-45f2-91d7-bedb8f428ad7","Type":"ContainerStarted","Data":"00488f536304b011c06340ab523500d32b7308fe36c24d9c921c58aadee20079"}
Jul 19 14:47:01.774712 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.774680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2vssd" event={"ID":"24aa3357-ff95-449c-b1b5-fda0fac5df4a","Type":"ContainerStarted","Data":"9d7f412051bfecbbcd9d24d5a8990e731ea944a9ffa6483f7e52b591d2c96ddc"}
Jul 19 14:47:01.799175 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.799149    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:47:01.803373 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.803346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal" event={"ID":"6e72960bc383aaed664cf3e1030c9130","Type":"ContainerStarted","Data":"48ca36427407616b93c94d7e008083a92e55b8c7f6664f158017c0ee64ed622d"}
Jul 19 14:47:01.955482 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:01.955404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:01.955659 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.955603    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:01.955718 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:01.955675    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:47:03.955654846 +0000 UTC m=+4.980663908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:02.056066 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:02.056019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:02.056248 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:02.056123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:02.056309 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.056272    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:02.056309 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.056291    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:02.056309 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.056305    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:02.056446 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.056371    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:04.056351191 +0000 UTC m=+5.081360248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:02.056818 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.056797    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:02.056929 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.056858    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:04.056843191 +0000 UTC m=+5.081852248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:02.371131 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:02.371011    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:42:00 +0000 UTC" deadline="2028-02-06 15:56:57.244098913 +0000 UTC"
Jul 19 14:47:02.371131 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:02.371052    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13609h9m54.873050865s"
Jul 19 14:47:02.534747 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:02.534159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:02.534747 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:02.534292    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:03.545130 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:03.544663    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:03.545130 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:03.544786    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:03.545130 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:03.545039    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:03.545738 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:03.545196    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:03.972576 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:03.972528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:03.972760 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:03.972743    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:03.972822 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:03.972813    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:47:07.972792603 +0000 UTC m=+8.997801658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:04.073466 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:04.073428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:04.073643 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:04.073487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:04.073703 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.073634    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:04.073703 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.073661    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:04.073703 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.073675    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:04.073838 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.073736    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:08.073716451 +0000 UTC m=+9.098725507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:04.074069 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.073641    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:04.074152 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.074105    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:08.074090855 +0000 UTC m=+9.099099909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:04.533764 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:04.533712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:04.533951 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:04.533864    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:05.535165 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:05.534621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:05.535165 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:05.534630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:05.535165 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:05.534762    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:05.535165 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:05.534824    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:06.533638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:06.533601    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:06.533817 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:06.533755    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:07.533235 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:07.533205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:07.533235 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:07.533205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:07.533767 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:07.533348    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:07.533767 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:07.533622    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:08.006336 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:08.006297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:08.006530 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.006467    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:08.007085 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.006728    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:47:16.006701256 +0000 UTC m=+17.031710313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:08.107673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:08.107640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:08.107844 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.107794    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:08.107844 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:08.107810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:08.107936 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.107857    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:16.107839083 +0000 UTC m=+17.132848149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:08.108257 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.108229    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:08.108257 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.108258    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:08.108442 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.108272    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:08.108442 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.108347    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:16.108332321 +0000 UTC m=+17.133341377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:08.533329 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:08.533301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:08.533808 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:08.533421    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:09.534834 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:09.534801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:09.535295 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:09.534903    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:09.535295 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:09.535236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:09.535483 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:09.535319    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:10.533916 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:10.533879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:10.534098 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:10.534030    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:11.533848 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:11.533751    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:11.534255 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:11.533891    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:11.534255 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:11.533948    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:11.534255 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:11.534052    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:12.533818 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:12.533791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:12.533930 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:12.533909    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:13.534048 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:13.534015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:13.534513 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:13.534150    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:13.534513 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:13.534187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:13.534513 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:13.534246    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:14.533584 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:14.533553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:14.533765 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:14.533675    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:15.533929 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:15.533897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:15.534394 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:15.533905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:15.534394 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:15.534027    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:15.534394 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:15.534115    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:16.069749 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:16.069716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:16.069955 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.069831    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:16.069955 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.069895    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:47:32.069880457 +0000 UTC m=+33.094889531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:16.170391 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:16.170356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:16.170557 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:16.170407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:16.170557 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.170543    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:16.170634 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.170563    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:16.170671 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.170648    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:32.170625613 +0000 UTC m=+33.195634689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:16.170671 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.170569    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:16.170671 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.170670    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:16.170787 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.170713    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:32.170701756 +0000 UTC m=+33.195710809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:16.533957 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:16.533884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:16.534453 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:16.534027    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:17.533806 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:17.533669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:17.533806 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:17.533689    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:17.533806 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:17.533804    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:17.534375 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:17.534218    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:18.533222 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:18.533189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:18.533399 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:18.533332    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:19.534171 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:19.534138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:19.534638 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:19.534255    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:19.534638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:19.534300    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:19.534638 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:19.534413    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:20.533072 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:20.533044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:20.533224 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:20.533147    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:21.533508 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.533226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:21.534090 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.533281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:21.534090 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:21.533649    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:21.534090 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:21.533644    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:21.859042 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.859010    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"df344348f07e292aa720be71c6be7f7db9e6af8a80abf80996956efacc733ebc"}
Jul 19 14:47:21.859198 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.859093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"1e6f4204103a3e9b9362de5b7d2d64cf28512b412deb62b2b5d852fa0f7b7016"}
Jul 19 14:47:21.859198 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.859142    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"865029b7608778cefdc5967cd8aaa25d48afe3dc4a3168e7802c4ba55bdc0637"}
Jul 19 14:47:21.859198 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.859186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"46196d7d1c87e9d3c5e5f4d260c13a904d95ac40e8eca2bf911d83cab6bda789"}
Jul 19 14:47:21.859363 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.859228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"b050c38f146fa7f261921588c589f977ca13457d06e6368dc334079ace60151c"}
Jul 19 14:47:21.859363 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.859273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"f128ec756c868a2f13240327d8eed537b3c0bb219c386aaa40be06ae4b4a7f65"}
Jul 19 14:47:21.860568 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.860542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s4nr8" event={"ID":"e4d95ccc-1c36-4ac7-b655-937995dd14ac","Type":"ContainerStarted","Data":"a4795ea1e8bd6352de0f27b08975128ea85b859e93231c8702e1733e9cfbccb6"}
Jul 19 14:47:21.862231 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.862198    2577 generic.go:356] "Generic (PLEG): container finished" podID="da3846eb-8ab6-4170-bba9-5f7c0886b7c4" containerID="ffd6c5b26c9625c4bfbf1bda63e2a9718dc25633a823534144bbae7a76652dd5" exitCode=0
Jul 19 14:47:21.862335 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.862254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerDied","Data":"ffd6c5b26c9625c4bfbf1bda63e2a9718dc25633a823534144bbae7a76652dd5"}
Jul 19 14:47:21.863902 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.863879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wxdbs" event={"ID":"fa88057d-da49-49e6-ac70-a17af00c1060","Type":"ContainerStarted","Data":"38ffc492236b24ba39b8b3f013a9f4e55ee7fe3aa985895e77f4adb748f32085"}
Jul 19 14:47:21.865479 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.865449    2577 generic.go:356] "Generic (PLEG): container finished" podID="49b4d8c3647ee516d9bd0bef4be2f438" containerID="fda020d959f78d325a83342c120b12521f80f197e4f744a3b3825c91c0fa151b" exitCode=0
Jul 19 14:47:21.865592 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.865540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal" event={"ID":"49b4d8c3647ee516d9bd0bef4be2f438","Type":"ContainerDied","Data":"fda020d959f78d325a83342c120b12521f80f197e4f744a3b3825c91c0fa151b"}
Jul 19 14:47:21.867122 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.867099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7n5n2" event={"ID":"9af30ba0-f27d-45f2-91d7-bedb8f428ad7","Type":"ContainerStarted","Data":"e548ca081fbe8e6a888b0258f12c6c61d755471900601d6818a1feffce8d6268"}
Jul 19 14:47:21.868940 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.868658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2vssd" event={"ID":"24aa3357-ff95-449c-b1b5-fda0fac5df4a","Type":"ContainerStarted","Data":"819799f73c046169fe7d1ab53160b7fb20a84bb33994e1394e3804b1e747999b"}
Jul 19 14:47:21.870224 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.870201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal" event={"ID":"6e72960bc383aaed664cf3e1030c9130","Type":"ContainerStarted","Data":"e8a6bdbc387c0cde4765bb4d2d360c8b5c5d6d56a9493f29ca6da2b3766bff95"}
Jul 19 14:47:21.871849 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.871820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2xxhx" event={"ID":"d09e1cc9-a996-410f-aee1-833b56eeda4c","Type":"ContainerStarted","Data":"e44d321c2d10f36707e00d412a5badbf39d66f3fe260a52d5501b11a442ac544"}
Jul 19 14:47:21.873234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.873216    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n" event={"ID":"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab","Type":"ContainerStarted","Data":"f37cedfe343d27eb6b813e2aac51801249fb05b637e8d80cb46751394e230072"}
Jul 19 14:47:21.887000 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.886959    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s4nr8" podStartSLOduration=2.6877690039999997 podStartE2EDuration="22.886946485s" totalImagesPullingTime="20.199177481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:21.886567679 +0000 UTC m=+22.911576754" watchObservedRunningTime="2026-07-19 14:47:21.886946485 +0000 UTC m=+22.911955560"
Jul 19 14:47:21.911784 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.911748    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2vssd" podStartSLOduration=2.877101799 podStartE2EDuration="22.911736141s" totalImagesPullingTime="20.034634342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:21.911567549 +0000 UTC m=+22.936576623" watchObservedRunningTime="2026-07-19 14:47:21.911736141 +0000 UTC m=+22.936745213"
Jul 19 14:47:21.988864 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:21.988808    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wxdbs" podStartSLOduration=2.796029634 podStartE2EDuration="22.988793961s" totalImagesPullingTime="20.192764327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:21.941635742 +0000 UTC m=+22.966644816" watchObservedRunningTime="2026-07-19 14:47:21.988793961 +0000 UTC m=+23.013803037"
Jul 19 14:47:22.013895 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.013851    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2xxhx" podStartSLOduration=3.257941468 podStartE2EDuration="23.013832776s" totalImagesPullingTime="19.755891308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:22.013392292 +0000 UTC m=+23.038401367" watchObservedRunningTime="2026-07-19 14:47:22.013832776 +0000 UTC m=+23.038841852"
Jul 19 14:47:22.030759 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.030601    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7n5n2" podStartSLOduration=3.152418286 podStartE2EDuration="23.030589419s" totalImagesPullingTime="19.878171133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:22.030393239 +0000 UTC m=+23.055402313" watchObservedRunningTime="2026-07-19 14:47:22.030589419 +0000 UTC m=+23.055598493"
Jul 19 14:47:22.046928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.046896    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-137.ec2.internal" podStartSLOduration=23.046883363 podStartE2EDuration="23.046883363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:22.046717039 +0000 UTC m=+23.071726115" watchObservedRunningTime="2026-07-19 14:47:22.046883363 +0000 UTC m=+23.071892420"
Jul 19 14:47:22.081145 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.081068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:22.156928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.156899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:22.533540 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.533519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:22.533886 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:22.533652    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:22.668287 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.668254    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 19 14:47:22.877831 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.877754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n" event={"ID":"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab","Type":"ContainerStarted","Data":"70966cec8525943166100888fa7fa700611fa46b416b761ce818400247bab11c"}
Jul 19 14:47:22.879087 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.879049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7jxc" event={"ID":"4fcdb84c-934b-466b-874b-d85c7a45b15c","Type":"ContainerStarted","Data":"11da09b35863996ffa751be5a1646c6756bdc739d6854acd9bba384e51753001"}
Jul 19 14:47:22.892131 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.892094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r7jxc" podStartSLOduration=3.837214411 podStartE2EDuration="23.892082677s" totalImagesPullingTime="20.054868266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:22.891724992 +0000 UTC m=+23.916734064" watchObservedRunningTime="2026-07-19 14:47:22.892082677 +0000 UTC m=+23.917091751"
Jul 19 14:47:22.985156 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:22.985120    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:23.085871 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.085841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:23.452359 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.452160    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-19T14:47:22.668274849Z","UUID":"9ad86be3-bc58-4c9a-92ca-ee203abd5c81","Handler":null,"Name":"","Endpoint":""}
Jul 19 14:47:23.453871 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.453848    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 19 14:47:23.454000 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.453882    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 19 14:47:23.533433 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.533394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:23.533613 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.533397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:23.533613 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:23.533544    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:23.533613 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:23.533582    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:23.882551 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.882526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal" event={"ID":"49b4d8c3647ee516d9bd0bef4be2f438","Type":"ContainerStarted","Data":"650d85bd9d67c08e52addeb3ceb2f97ac34b7e24f94a18dc7e6339f53a108302"}
Jul 19 14:47:23.904814 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:23.904777    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-137.ec2.internal" podStartSLOduration=24.904764278000002 podStartE2EDuration="24.904764278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:23.904514696 +0000 UTC m=+24.929523763" watchObservedRunningTime="2026-07-19 14:47:23.904764278 +0000 UTC m=+24.929773352"
Jul 19 14:47:24.533700 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:24.533664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:24.534182 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:24.533808    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:24.887896 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:24.887813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n" event={"ID":"6c6c3cbb-8e2b-47a5-9118-73680d9a11ab","Type":"ContainerStarted","Data":"609ead57997fb680db49ae4265dd9cd977f7e834ae2f72cbe8cefafb3f8ed83e"}
Jul 19 14:47:24.891618 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:24.891593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"a51e13bce9552839d8e3a389072cb74e11c2964a32b3ea8eefe88ca3e169f9d5"}
Jul 19 14:47:24.905800 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:24.905746    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgd4n" podStartSLOduration=3.541671359 podStartE2EDuration="25.905727346s" totalImagesPullingTime="22.364055987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:24.903986833 +0000 UTC m=+25.928995907" watchObservedRunningTime="2026-07-19 14:47:24.905727346 +0000 UTC m=+25.930736421"
Jul 19 14:47:25.533292 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:25.533252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:25.533292 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:25.533287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:25.533555 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:25.533382    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:25.533555 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:25.533534    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:26.108906 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.108868    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:26.109629 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.109603    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:26.533308 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.533282    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:26.533466 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:26.533384    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:26.898083 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.897910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" event={"ID":"decd7f26-f653-4505-a3d1-f8a4805c792a","Type":"ContainerStarted","Data":"a1dea6906d1cfd450979c4c9ad963152c791fdf9505cb09f1d119d208fa69fbc"}
Jul 19 14:47:26.898229 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.898175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:26.898229 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.898223    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:26.898329 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.898253    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:26.899815 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.899783    2577 generic.go:356] "Generic (PLEG): container finished" podID="da3846eb-8ab6-4170-bba9-5f7c0886b7c4" containerID="addac02c225b4656ce9328a67c7348fd77b3469e62413036c53b0d92d2be23f3" exitCode=0
Jul 19 14:47:26.899922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.899862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerDied","Data":"addac02c225b4656ce9328a67c7348fd77b3469e62413036c53b0d92d2be23f3"}
Jul 19 14:47:26.900003 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.899988    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:26.900558 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.900521    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2vssd"
Jul 19 14:47:26.913383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.913359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:26.913464 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.913456    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:26.928366 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.928317    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh" podStartSLOduration=7.835470228 podStartE2EDuration="27.928305271s" totalImagesPullingTime="20.092835043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:26.928042014 +0000 UTC m=+27.953051089" watchObservedRunningTime="2026-07-19 14:47:26.928305271 +0000 UTC m=+27.953314345"
Jul 19 14:47:26.989132 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:26.989105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:27.536039 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:27.536015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:27.536039 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:27.536039    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:27.536753 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:27.536107    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:27.536753 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:27.536239    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:28.533880 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.533711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:28.534075 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:28.533975    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:28.546290 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.546233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kpllx"]
Jul 19 14:47:28.546673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.546327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:28.546673 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:28.546411    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:28.547118 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.547093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pdcdf"]
Jul 19 14:47:28.547768 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.547743    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-svvdc"]
Jul 19 14:47:28.547871 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.547846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:28.547957 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:28.547937    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:28.905525 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.905438    2577 generic.go:356] "Generic (PLEG): container finished" podID="da3846eb-8ab6-4170-bba9-5f7c0886b7c4" containerID="e60a655a2769dd63632a632da83f74f6d8332120d09b26964ad9de4dc245ef64" exitCode=0
Jul 19 14:47:28.905669 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.905543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerDied","Data":"e60a655a2769dd63632a632da83f74f6d8332120d09b26964ad9de4dc245ef64"}
Jul 19 14:47:28.905669 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:28.905587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:28.905813 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:28.905788    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:30.533262 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:30.533232    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:30.533730 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:30.533236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:30.533730 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:30.533333    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:30.533730 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:30.533356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:30.533730 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:30.533452    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:30.533730 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:30.533529    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:30.912584 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:30.912489    2577 generic.go:356] "Generic (PLEG): container finished" podID="da3846eb-8ab6-4170-bba9-5f7c0886b7c4" containerID="a10cf0d3ff2129f197d771a40cec01702e054ea905add3b5d12f4231f8e2970d" exitCode=0
Jul 19 14:47:30.912584 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:30.912545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerDied","Data":"a10cf0d3ff2129f197d771a40cec01702e054ea905add3b5d12f4231f8e2970d"}
Jul 19 14:47:32.099677 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:32.099485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:32.100107 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.099654    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:32.100107 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.099763    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret podName:bd683c25-0c79-4e63-b8ba-6d1b14e2463f nodeName:}" failed. No retries permitted until 2026-07-19 14:48:04.099742125 +0000 UTC m=+65.124751191 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret") pod "global-pull-secret-syncer-kpllx" (UID: "bd683c25-0c79-4e63-b8ba-6d1b14e2463f") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:32.200574 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:32.200502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:32.200574 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:32.200542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:32.200757 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.200640    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:32.200757 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.200666    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:32.200757 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.200692    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:32.200757 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.200705    2577 projected.go:196] Error preparing data for projected volume kube-api-access-679n8 for pod openshift-network-diagnostics/network-check-target-svvdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:32.200757 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.200694    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:48:04.200676293 +0000 UTC m=+65.225685349 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:32.200915 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.200763    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8 podName:4f5624bc-1a72-4a1d-8bc1-589ee9531480 nodeName:}" failed. No retries permitted until 2026-07-19 14:48:04.200746171 +0000 UTC m=+65.225755228 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-679n8" (UniqueName: "kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8") pod "network-check-target-svvdc" (UID: "4f5624bc-1a72-4a1d-8bc1-589ee9531480") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:32.534005 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:32.533924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:32.534005 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:32.533988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:32.534227 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:32.533988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:32.534227 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.534144    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:47:32.534227 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.534180    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kpllx" podUID="bd683c25-0c79-4e63-b8ba-6d1b14e2463f"
Jul 19 14:47:32.534375 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:32.534254    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svvdc" podUID="4f5624bc-1a72-4a1d-8bc1-589ee9531480"
Jul 19 14:47:33.801692 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.801659    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-137.ec2.internal" event="NodeReady"
Jul 19 14:47:33.802155 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.801812    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 19 14:47:33.848402 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.848362    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8rllc"]
Jul 19 14:47:33.855107 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.855084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:33.855487 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.855457    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d5lrq"]
Jul 19 14:47:33.857539 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.857512    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 19 14:47:33.857659 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.857587    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4r258\""
Jul 19 14:47:33.857885 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.857866    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 19 14:47:33.858568 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.858549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:33.860739 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.860721    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 19 14:47:33.860845 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.860824    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8987\""
Jul 19 14:47:33.860939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.860924    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 19 14:47:33.861123 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.861099    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 19 14:47:33.863888 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.863832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8rllc"]
Jul 19 14:47:33.866577 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.866558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d5lrq"]
Jul 19 14:47:33.984019 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:33.983990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:34.016205 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.016174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p74gn\" (UniqueName: \"kubernetes.io/projected/7af39c6e-cb98-4b68-a6de-074e52edfd22-kube-api-access-p74gn\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.016348 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.016226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6mnv\" (UniqueName: \"kubernetes.io/projected/fd583199-d177-4296-a397-e3f9e45ac406-kube-api-access-j6mnv\") pod \"ingress-canary-d5lrq\" (UID: \"fd583199-d177-4296-a397-e3f9e45ac406\") " pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.016348 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.016286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7af39c6e-cb98-4b68-a6de-074e52edfd22-config-volume\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.016348 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.016320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7af39c6e-cb98-4b68-a6de-074e52edfd22-tmp-dir\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.016472 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.016354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd583199-d177-4296-a397-e3f9e45ac406-cert\") pod \"ingress-canary-d5lrq\" (UID: \"fd583199-d177-4296-a397-e3f9e45ac406\") " pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.016472 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.016389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.046313 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.046263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:34.117585 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.117490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p74gn\" (UniqueName: \"kubernetes.io/projected/7af39c6e-cb98-4b68-a6de-074e52edfd22-kube-api-access-p74gn\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.117585 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.117568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6mnv\" (UniqueName: \"kubernetes.io/projected/fd583199-d177-4296-a397-e3f9e45ac406-kube-api-access-j6mnv\") pod \"ingress-canary-d5lrq\" (UID: \"fd583199-d177-4296-a397-e3f9e45ac406\") " pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.117585 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.117585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7af39c6e-cb98-4b68-a6de-074e52edfd22-config-volume\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.117836 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.117605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7af39c6e-cb98-4b68-a6de-074e52edfd22-tmp-dir\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.117836 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.117632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd583199-d177-4296-a397-e3f9e45ac406-cert\") pod \"ingress-canary-d5lrq\" (UID: \"fd583199-d177-4296-a397-e3f9e45ac406\") " pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.117836 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.117656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.117836 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:34.117803    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:34.118028 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:34.117853    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:34.617835512 +0000 UTC m=+35.642844578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:34.118095 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.118076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7af39c6e-cb98-4b68-a6de-074e52edfd22-tmp-dir\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.118303 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.118267    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7af39c6e-cb98-4b68-a6de-074e52edfd22-config-volume\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.122392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.122369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd583199-d177-4296-a397-e3f9e45ac406-cert\") pod \"ingress-canary-d5lrq\" (UID: \"fd583199-d177-4296-a397-e3f9e45ac406\") " pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.126718 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.126693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6mnv\" (UniqueName: \"kubernetes.io/projected/fd583199-d177-4296-a397-e3f9e45ac406-kube-api-access-j6mnv\") pod \"ingress-canary-d5lrq\" (UID: \"fd583199-d177-4296-a397-e3f9e45ac406\") " pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.127121 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.127092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p74gn\" (UniqueName: \"kubernetes.io/projected/7af39c6e-cb98-4b68-a6de-074e52edfd22-kube-api-access-p74gn\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.176183 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.176149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d5lrq"
Jul 19 14:47:34.533266 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.533224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:47:34.533266 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.533265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:47:34.533535 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.533265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:47:34.536165 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.536142    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:47:34.537083 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.537063    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c6kqk\""
Jul 19 14:47:34.537485 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.537459    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:47:34.537588 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.537510    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:47:34.537588 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.537527    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:47:34.537812 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.537797    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5p5rq\""
Jul 19 14:47:34.622663 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:34.622638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:34.622813 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:34.622760    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:34.622891 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:34.622829    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:35.622809886 +0000 UTC m=+36.647818939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:35.182785 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:35.182594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d5lrq"]
Jul 19 14:47:35.186568 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:47:35.186545    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd583199_d177_4296_a397_e3f9e45ac406.slice/crio-5cf18c169265b27127abb19951a6dec320b3b3d7c55e6c1a17ed8548068afa45 WatchSource:0}: Error finding container 5cf18c169265b27127abb19951a6dec320b3b3d7c55e6c1a17ed8548068afa45: Status 404 returned error can't find the container with id 5cf18c169265b27127abb19951a6dec320b3b3d7c55e6c1a17ed8548068afa45
Jul 19 14:47:35.631269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:35.631237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:35.631467 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:35.631405    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:35.631541 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:35.631475    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:37.631457765 +0000 UTC m=+38.656466830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:35.927888 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:35.927805    2577 generic.go:356] "Generic (PLEG): container finished" podID="da3846eb-8ab6-4170-bba9-5f7c0886b7c4" containerID="8a9cd7625cde785475f9ca0348e7232856bdcdf05ae738819e21478506842f37" exitCode=0
Jul 19 14:47:35.928053 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:35.927889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerDied","Data":"8a9cd7625cde785475f9ca0348e7232856bdcdf05ae738819e21478506842f37"}
Jul 19 14:47:35.929122 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:35.929096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d5lrq" event={"ID":"fd583199-d177-4296-a397-e3f9e45ac406","Type":"ContainerStarted","Data":"5cf18c169265b27127abb19951a6dec320b3b3d7c55e6c1a17ed8548068afa45"}
Jul 19 14:47:36.935809 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:36.935777    2577 generic.go:356] "Generic (PLEG): container finished" podID="da3846eb-8ab6-4170-bba9-5f7c0886b7c4" containerID="7f78e61a975203d2f0d3dee4668a970161219b059ef22a4b3ee1269b1618208a" exitCode=0
Jul 19 14:47:36.936204 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:36.935822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerDied","Data":"7f78e61a975203d2f0d3dee4668a970161219b059ef22a4b3ee1269b1618208a"}
Jul 19 14:47:37.648404 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.648377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:37.648564 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:37.648510    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:37.648564 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:37.648560    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:41.648547644 +0000 UTC m=+42.673556696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:37.921100 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.920888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:37.941407 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.941383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" event={"ID":"da3846eb-8ab6-4170-bba9-5f7c0886b7c4","Type":"ContainerStarted","Data":"7a6f8c961dd0fadb0ec2a16d19734a62a12fb125d80cb17da70939fccc73e609"}
Jul 19 14:47:37.942875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.942853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d5lrq" event={"ID":"fd583199-d177-4296-a397-e3f9e45ac406","Type":"ContainerStarted","Data":"44c1bed9737f23f75858d0b785101dc61aa3c38553fb49feb213542b71751ac7"}
Jul 19 14:47:37.964258 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.964208    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zcsxf" podStartSLOduration=7.811674101 podStartE2EDuration="38.964189067s" totalImagesPullingTime="31.152514966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:37.963026433 +0000 UTC m=+38.988035508" watchObservedRunningTime="2026-07-19 14:47:37.964189067 +0000 UTC m=+38.989198144"
Jul 19 14:47:37.983662 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.983616    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d5lrq" podStartSLOduration=2.5978852999999997 podStartE2EDuration="4.983599919s" totalImagesPullingTime="2.385714619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:37.9831209 +0000 UTC m=+39.008129975" watchObservedRunningTime="2026-07-19 14:47:37.983599919 +0000 UTC m=+39.008608993"
Jul 19 14:47:37.987699 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:37.987667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:38.057259 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:38.057186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:38.121465 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:38.121431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:39.092624 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:39.092588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:39.143486 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:39.143450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:40.188185 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:40.188150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:40.289267 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:40.289239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:41.679907 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:41.679864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:41.680260 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:41.680024    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:41.680260 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:41.680102    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:49.680086508 +0000 UTC m=+50.705095579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:49.736207 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:49.736170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:47:49.736660 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:49.736305    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:49.736660 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:47:49.736386    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:48:05.736370271 +0000 UTC m=+66.761379324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:58.925221 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:58.925192    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjgvh"
Jul 19 14:47:59.040072 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:59.040044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:59.140871 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:47:59.140844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:04.136460 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.136424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:48:04.139768 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.139748    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:48:04.150800 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.150772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd683c25-0c79-4e63-b8ba-6d1b14e2463f-original-pull-secret\") pod \"global-pull-secret-syncer-kpllx\" (UID: \"bd683c25-0c79-4e63-b8ba-6d1b14e2463f\") " pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:48:04.237620 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.237598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:48:04.237733 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.237634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:48:04.242210 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.242193    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:48:04.243345 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.243328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:48:04.246324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.246306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kpllx"
Jul 19 14:48:04.248428 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:48:04.248414    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:48:04.248514 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:48:04.248477    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:49:08.248457252 +0000 UTC m=+129.273466312 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : secret "metrics-daemon-secret" not found
Jul 19 14:48:04.252256 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.252241    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:48:04.261366 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.261339    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-679n8\" (UniqueName: \"kubernetes.io/projected/4f5624bc-1a72-4a1d-8bc1-589ee9531480-kube-api-access-679n8\") pod \"network-check-target-svvdc\" (UID: \"4f5624bc-1a72-4a1d-8bc1-589ee9531480\") " pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:48:04.366217 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.366188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kpllx"]
Jul 19 14:48:04.368928 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:48:04.368895    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd683c25_0c79_4e63_b8ba_6d1b14e2463f.slice/crio-586235ef97a0a062d8d8c3870200c37fa798f7aea66374408ee2fcee6a1d6482 WatchSource:0}: Error finding container 586235ef97a0a062d8d8c3870200c37fa798f7aea66374408ee2fcee6a1d6482: Status 404 returned error can't find the container with id 586235ef97a0a062d8d8c3870200c37fa798f7aea66374408ee2fcee6a1d6482
Jul 19 14:48:04.555277 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.555247    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c6kqk\""
Jul 19 14:48:04.563330 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.563309    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:48:04.685687 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:04.685658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-svvdc"]
Jul 19 14:48:04.688843 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:48:04.688807    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5624bc_1a72_4a1d_8bc1_589ee9531480.slice/crio-ed903c5b005de55f54a82560bf6c01ffe1550cdab650d606acd5af8729f1e446 WatchSource:0}: Error finding container ed903c5b005de55f54a82560bf6c01ffe1550cdab650d606acd5af8729f1e446: Status 404 returned error can't find the container with id ed903c5b005de55f54a82560bf6c01ffe1550cdab650d606acd5af8729f1e446
Jul 19 14:48:05.007456 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:05.007421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kpllx" event={"ID":"bd683c25-0c79-4e63-b8ba-6d1b14e2463f","Type":"ContainerStarted","Data":"586235ef97a0a062d8d8c3870200c37fa798f7aea66374408ee2fcee6a1d6482"}
Jul 19 14:48:05.008615 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:05.008586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-svvdc" event={"ID":"4f5624bc-1a72-4a1d-8bc1-589ee9531480","Type":"ContainerStarted","Data":"ed903c5b005de55f54a82560bf6c01ffe1550cdab650d606acd5af8729f1e446"}
Jul 19 14:48:05.749310 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:05.749274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:48:05.749828 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:48:05.749426    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:48:05.749828 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:48:05.749511    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:48:37.749476054 +0000 UTC m=+98.774485109 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:48:07.437755 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.437718    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"]
Jul 19 14:48:07.459832 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.459804    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"]
Jul 19 14:48:07.459990 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.459935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.463152 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.463117    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 19 14:48:07.463284 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.463204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 19 14:48:07.463284 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.463257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 19 14:48:07.463578 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.463553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 19 14:48:07.463664 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.463618    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bffhh\""
Jul 19 14:48:07.475039 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.475021    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"]
Jul 19 14:48:07.475157 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.475052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"]
Jul 19 14:48:07.475157 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.475134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.478091 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.478062    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 19 14:48:07.478091 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.478071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 19 14:48:07.478258 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.478122    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 19 14:48:07.478258 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.478071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 19 14:48:07.563108 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.562998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb05bf36-6d62-4e7a-a5db-948e9b0db49d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d8f54888-h6jgt\" (UID: \"eb05bf36-6d62-4e7a-a5db-948e9b0db49d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.563108 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-hub\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.563324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-ca\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.563324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnjlz\" (UniqueName: \"kubernetes.io/projected/eb05bf36-6d62-4e7a-a5db-948e9b0db49d-kube-api-access-gnjlz\") pod \"managed-serviceaccount-addon-agent-7d8f54888-h6jgt\" (UID: \"eb05bf36-6d62-4e7a-a5db-948e9b0db49d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.563324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlvx4\" (UniqueName: \"kubernetes.io/projected/e5cc234a-b64a-449e-8c43-4279feb986dd-kube-api-access-jlvx4\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.563324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.563324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.563324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.563320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cc234a-b64a-449e-8c43-4279feb986dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.664276 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-ca\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.664434 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnjlz\" (UniqueName: \"kubernetes.io/projected/eb05bf36-6d62-4e7a-a5db-948e9b0db49d-kube-api-access-gnjlz\") pod \"managed-serviceaccount-addon-agent-7d8f54888-h6jgt\" (UID: \"eb05bf36-6d62-4e7a-a5db-948e9b0db49d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.664434 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlvx4\" (UniqueName: \"kubernetes.io/projected/e5cc234a-b64a-449e-8c43-4279feb986dd-kube-api-access-jlvx4\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.664434 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.664434 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.664689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cc234a-b64a-449e-8c43-4279feb986dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.664689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb05bf36-6d62-4e7a-a5db-948e9b0db49d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d8f54888-h6jgt\" (UID: \"eb05bf36-6d62-4e7a-a5db-948e9b0db49d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.664689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.664522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-hub\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.665345 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.665235    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cc234a-b64a-449e-8c43-4279feb986dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.667302 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.667269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-ca\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.667535 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.667472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.667824 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.667803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-hub\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.667890 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.667803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb05bf36-6d62-4e7a-a5db-948e9b0db49d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d8f54888-h6jgt\" (UID: \"eb05bf36-6d62-4e7a-a5db-948e9b0db49d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.668164 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.668149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e5cc234a-b64a-449e-8c43-4279feb986dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.672574 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.672554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlvx4\" (UniqueName: \"kubernetes.io/projected/e5cc234a-b64a-449e-8c43-4279feb986dd-kube-api-access-jlvx4\") pod \"cluster-proxy-proxy-agent-57d48659d6-2chjp\" (UID: \"e5cc234a-b64a-449e-8c43-4279feb986dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:07.672574 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.672570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnjlz\" (UniqueName: \"kubernetes.io/projected/eb05bf36-6d62-4e7a-a5db-948e9b0db49d-kube-api-access-gnjlz\") pod \"managed-serviceaccount-addon-agent-7d8f54888-h6jgt\" (UID: \"eb05bf36-6d62-4e7a-a5db-948e9b0db49d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.779679 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.779646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"
Jul 19 14:48:07.786827 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:07.786800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:48:09.311756 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:09.311702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"]
Jul 19 14:48:09.315363 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:48:09.315324    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5cc234a_b64a_449e_8c43_4279feb986dd.slice/crio-37f0106be24f5127b7dbb78274c11ca55bc48964ef9c3f687a307e1bc9a79546 WatchSource:0}: Error finding container 37f0106be24f5127b7dbb78274c11ca55bc48964ef9c3f687a307e1bc9a79546: Status 404 returned error can't find the container with id 37f0106be24f5127b7dbb78274c11ca55bc48964ef9c3f687a307e1bc9a79546
Jul 19 14:48:09.322876 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:09.322842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt"]
Jul 19 14:48:09.326642 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:48:09.326619    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb05bf36_6d62_4e7a_a5db_948e9b0db49d.slice/crio-0b9488606d378aad563f2d4628d5a96632b3684b9f9490f7ba523351f662bc9e WatchSource:0}: Error finding container 0b9488606d378aad563f2d4628d5a96632b3684b9f9490f7ba523351f662bc9e: Status 404 returned error can't find the container with id 0b9488606d378aad563f2d4628d5a96632b3684b9f9490f7ba523351f662bc9e
Jul 19 14:48:10.025569 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:10.025535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-svvdc" event={"ID":"4f5624bc-1a72-4a1d-8bc1-589ee9531480","Type":"ContainerStarted","Data":"394332995b1dcb9f77ddcc0e836a011beb5f2df919e2f248ffa3e994c97da066"}
Jul 19 14:48:10.025763 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:10.025659    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:48:10.026891 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:10.026866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kpllx" event={"ID":"bd683c25-0c79-4e63-b8ba-6d1b14e2463f","Type":"ContainerStarted","Data":"c9d6cdb30c8cf940b36ff15c7e4eda864895ac0a3de453aa5661d25afdca3548"}
Jul 19 14:48:10.027934 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:10.027915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" event={"ID":"e5cc234a-b64a-449e-8c43-4279feb986dd","Type":"ContainerStarted","Data":"37f0106be24f5127b7dbb78274c11ca55bc48964ef9c3f687a307e1bc9a79546"}
Jul 19 14:48:10.028910 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:10.028893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt" event={"ID":"eb05bf36-6d62-4e7a-a5db-948e9b0db49d","Type":"ContainerStarted","Data":"0b9488606d378aad563f2d4628d5a96632b3684b9f9490f7ba523351f662bc9e"}
Jul 19 14:48:10.040593 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:10.040553    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-svvdc" podStartSLOduration=66.577259033 podStartE2EDuration="1m11.040540639s" totalImagesPullingTime="4.463281606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:10.039577008 +0000 UTC m=+71.064586085" watchObservedRunningTime="2026-07-19 14:48:10.040540639 +0000 UTC m=+71.065549713"
Jul 19 14:48:14.042213 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.042170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" event={"ID":"e5cc234a-b64a-449e-8c43-4279feb986dd","Type":"ContainerStarted","Data":"ae326e061b2ad30c4f8d3fc865cc3d7a7a0e42aad0e65dbd94550e4b7fda5def"}
Jul 19 14:48:14.042667 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.042228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" event={"ID":"e5cc234a-b64a-449e-8c43-4279feb986dd","Type":"ContainerStarted","Data":"c5700a0708c3dbf00b5da2a6675faa56a845d867c4382275fc9eba2b3df956ef"}
Jul 19 14:48:14.042667 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.042247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" event={"ID":"e5cc234a-b64a-449e-8c43-4279feb986dd","Type":"ContainerStarted","Data":"68bd6c363139b158e9726e990c4308deb1f40644a9c6abe761072d0d3af0645e"}
Jul 19 14:48:14.043532 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.043511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt" event={"ID":"eb05bf36-6d62-4e7a-a5db-948e9b0db49d","Type":"ContainerStarted","Data":"3f3d2467350fdd0691cd7d781a0a1f85980372cc0ca7c6c33890707f46efbf26"}
Jul 19 14:48:14.059730 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.059691    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kpllx" podStartSLOduration=70.278367294 podStartE2EDuration="1m15.059678566s" totalImagesPullingTime="4.781311272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:10.05733426 +0000 UTC m=+71.082343335" watchObservedRunningTime="2026-07-19 14:48:14.059678566 +0000 UTC m=+75.084687640"
Jul 19 14:48:14.059925 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.059907    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" podStartSLOduration=3.136980543 podStartE2EDuration="7.059902705s" totalImagesPullingTime="3.922922162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:14.058717917 +0000 UTC m=+75.083726993" watchObservedRunningTime="2026-07-19 14:48:14.059902705 +0000 UTC m=+75.084911780"
Jul 19 14:48:14.076725 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:14.076688    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt" podStartSLOduration=3.173607969 podStartE2EDuration="7.076676506s" totalImagesPullingTime="3.903068537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:14.076565764 +0000 UTC m=+75.101574840" watchObservedRunningTime="2026-07-19 14:48:14.076676506 +0000 UTC m=+75.101685572"
Jul 19 14:48:37.773241 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:37.773208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:48:37.773723 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:48:37.773326    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:48:37.773723 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:48:37.773388    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:49:41.773374541 +0000 UTC m=+162.798383597 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:48:41.034121 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:48:41.034091    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-svvdc"
Jul 19 14:49:08.266671 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:49:08.266633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:49:08.267079 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:49:08.266769    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:49:08.267079 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:49:08.266846    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:10.266829664 +0000 UTC m=+251.291838716 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : secret "metrics-daemon-secret" not found
Jul 19 14:49:36.869678 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:49:36.869636    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8rllc" podUID="7af39c6e-cb98-4b68-a6de-074e52edfd22"
Jul 19 14:49:37.279475 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:49:37.279448    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8rllc"
Jul 19 14:49:37.558870 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:49:37.558797    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:49:41.779362 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:49:41.779327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:49:41.779764 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:49:41.779443    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:49:41.779764 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:49:41.779519    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls podName:7af39c6e-cb98-4b68-a6de-074e52edfd22 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:43.779489311 +0000 UTC m=+284.804498364 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls") pod "dns-default-8rllc" (UID: "7af39c6e-cb98-4b68-a6de-074e52edfd22") : secret "dns-default-metrics-tls" not found
Jul 19 14:49:50.219698 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:49:50.219667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:50.533934 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:49:50.533904    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:50:13.384407 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:50:13.384378    2577 generic.go:356] "Generic (PLEG): container finished" podID="eb05bf36-6d62-4e7a-a5db-948e9b0db49d" containerID="3f3d2467350fdd0691cd7d781a0a1f85980372cc0ca7c6c33890707f46efbf26" exitCode=255
Jul 19 14:50:13.384759 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:50:13.384451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt" event={"ID":"eb05bf36-6d62-4e7a-a5db-948e9b0db49d","Type":"ContainerDied","Data":"3f3d2467350fdd0691cd7d781a0a1f85980372cc0ca7c6c33890707f46efbf26"}
Jul 19 14:50:13.384759 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:50:13.384730    2577 scope.go:117] "RemoveContainer" containerID="3f3d2467350fdd0691cd7d781a0a1f85980372cc0ca7c6c33890707f46efbf26"
Jul 19 14:50:14.389255 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:50:14.389222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d8f54888-h6jgt" event={"ID":"eb05bf36-6d62-4e7a-a5db-948e9b0db49d","Type":"ContainerStarted","Data":"8e1815b4ff7fb4c7e151fbca24bb0861b1edb467d2d0702ad9445af46d6171af"}
Jul 19 14:50:37.493219 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:50:37.493182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:02.093340 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:02.093304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:02.192726 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:02.192696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:04.607538 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:04.607482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:04.708831 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:04.708795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:05.950694 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:05.950663    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:06.004862 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.004834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:06.625003 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.624972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:06.733888 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.733857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:06.844570 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.844547    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t5xjc"]
Jul 19 14:51:06.846411 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.846396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:06.849822 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.849803    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 19 14:51:06.850721 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.850702    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 19 14:51:06.850814 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.850754    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 19 14:51:06.850869 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.850819    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 19 14:51:06.850869 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.850828    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gskh7\""
Jul 19 14:51:06.858657 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.858637    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5xjc"]
Jul 19 14:51:06.862883 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.862862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:06.923666 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.923604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khscl\" (UniqueName: \"kubernetes.io/projected/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-api-access-khscl\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:06.923782 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.923666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77208df7-adea-4cbd-88fb-9444a27a1afa-crio-socket\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:06.923782 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.923689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:06.923782 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.923706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:06.923782 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.923729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77208df7-adea-4cbd-88fb-9444a27a1afa-data-volume\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:06.929269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:06.929244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:07.024865 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.024840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77208df7-adea-4cbd-88fb-9444a27a1afa-crio-socket\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.024871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.024892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.024914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77208df7-adea-4cbd-88fb-9444a27a1afa-data-volume\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.024948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khscl\" (UniqueName: \"kubernetes.io/projected/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-api-access-khscl\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.024953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77208df7-adea-4cbd-88fb-9444a27a1afa-crio-socket\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.024970    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.025019    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.025038    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:07.525020664 +0000 UTC m=+248.550029719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.025220 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.025077    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:07.525062041 +0000 UTC m=+248.550071108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:07.025566 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.025269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77208df7-adea-4cbd-88fb-9444a27a1afa-data-volume\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.032781 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.032763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khscl\" (UniqueName: \"kubernetes.io/projected/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-api-access-khscl\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.530060 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.530030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.530060 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:07.530063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:07.530311 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.530149    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.530311 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.530207    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:08.530191454 +0000 UTC m=+249.555200506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.530311 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.530234    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:07.530311 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:07.530285    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:08.530272517 +0000 UTC m=+249.555281587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:08.222387 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.222351    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-jwzbw"]
Jul 19 14:51:08.224126 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.224109    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.226403 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.226367    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 19 14:51:08.226550 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.226452    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 19 14:51:08.226550 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.226469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 19 14:51:08.227310 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.227288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 19 14:51:08.227848 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.227829    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bz7k7\""
Jul 19 14:51:08.231937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.231914    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-jwzbw"]
Jul 19 14:51:08.254870 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.254846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:08.336384 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.336360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb2cs\" (UniqueName: \"kubernetes.io/projected/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-kube-api-access-mb2cs\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.336479 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.336389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-signing-key\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.336479 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.336444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-signing-cabundle\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.437007 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.436986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-signing-cabundle\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.437118 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.437027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb2cs\" (UniqueName: \"kubernetes.io/projected/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-kube-api-access-mb2cs\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.437118 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.437044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-signing-key\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.437696 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.437676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-signing-cabundle\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.439382 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.439361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-signing-key\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.444612 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.444591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb2cs\" (UniqueName: \"kubernetes.io/projected/a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b-kube-api-access-mb2cs\") pod \"service-ca-58598695f7-jwzbw\" (UID: \"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b\") " pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.537128 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.537107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-jwzbw"
Jul 19 14:51:08.538355 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.538339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:08.538405 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.538366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:08.538451 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:08.538437    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:08.538490 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:08.538452    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:08.538543 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:08.538520    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:10.53848842 +0000 UTC m=+251.563497472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:08.538543 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:08.538537    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:10.538527915 +0000 UTC m=+251.563536982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:08.653148 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:08.653117    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-jwzbw"]
Jul 19 14:51:08.655675 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:51:08.655649    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cdeb86_14f2_42fb_8d9b_0fe1e010e60b.slice/crio-1b708f73ca78e3e6ddba113a013d05cd0f6223a568b6d91de16456863b3ec17e WatchSource:0}: Error finding container 1b708f73ca78e3e6ddba113a013d05cd0f6223a568b6d91de16456863b3ec17e: Status 404 returned error can't find the container with id 1b708f73ca78e3e6ddba113a013d05cd0f6223a568b6d91de16456863b3ec17e
Jul 19 14:51:09.038304 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:09.038280    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wxdbs_fa88057d-da49-49e6-ac70-a17af00c1060/dns-node-resolver/0.log"
Jul 19 14:51:09.085326 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:09.085298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:09.537959 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:09.537915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-jwzbw" event={"ID":"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b","Type":"ContainerStarted","Data":"1b708f73ca78e3e6ddba113a013d05cd0f6223a568b6d91de16456863b3ec17e"}
Jul 19 14:51:09.637938 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:09.637908    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7n5n2_9af30ba0-f27d-45f2-91d7-bedb8f428ad7/node-ca/0.log"
Jul 19 14:51:10.353560 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:10.353526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:51:10.353723 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:10.353666    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:51:10.353763 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:10.353731    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs podName:89704d43-d416-4b1b-853c-52ecefa8b7a1 nodeName:}" failed. No retries permitted until 2026-07-19 14:53:12.353712236 +0000 UTC m=+373.378721289 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs") pod "network-metrics-daemon-pdcdf" (UID: "89704d43-d416-4b1b-853c-52ecefa8b7a1") : secret "metrics-daemon-secret" not found
Jul 19 14:51:10.542789 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:10.542756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-jwzbw" event={"ID":"a5cdeb86-14f2-42fb-8d9b-0fe1e010e60b","Type":"ContainerStarted","Data":"420c9364c8f9c62267c2aab15dc8b511779569f2377fd08fa65eee10a481e6c0"}
Jul 19 14:51:10.555639 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:10.555613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:10.555738 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:10.555644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:10.555738 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:10.555723    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:10.555815 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:10.555752    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:10.555815 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:10.555777    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:14.555762924 +0000 UTC m=+255.580771976 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:10.555815 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:10.555797    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:14.555783455 +0000 UTC m=+255.580792508 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:10.557393 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:10.557356    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-jwzbw" podStartSLOduration=0.856365868 podStartE2EDuration="2.557342693s" totalImagesPullingTime="1.700976825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:10.556478419 +0000 UTC m=+251.581487503" watchObservedRunningTime="2026-07-19 14:51:10.557342693 +0000 UTC m=+251.582352033"
Jul 19 14:51:10.837382 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:10.837355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d5lrq_fd583199-d177-4296-a397-e3f9e45ac406/serve-healthcheck-canary/0.log"
Jul 19 14:51:11.445167 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:11.445109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:12.002799 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:12.002755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:12.129860 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:12.129827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:12.230246 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:12.230214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:12.343002 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:12.342864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:12.448053 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:12.448018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:12.883510 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:12.883468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:13.685757 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:13.685730    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:13.887186 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:13.887153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:14.098842 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:14.098809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:14.480706 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:14.480673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:14.587158 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:14.587113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:14.587158 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:14.587161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:14.587368 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:14.587231    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:14.587368 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:14.587262    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:14.587368 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:14.587292    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:22.587277306 +0000 UTC m=+263.612286362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:14.587368 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:14.587306    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls podName:77208df7-adea-4cbd-88fb-9444a27a1afa nodeName:}" failed. No retries permitted until 2026-07-19 14:51:22.587300463 +0000 UTC m=+263.612309516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5xjc" (UID: "77208df7-adea-4cbd-88fb-9444a27a1afa") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:14.883235 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:14.883152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:15.684545 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:15.684515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:15.737625 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:15.737599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:15.799335 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:15.799309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:15.885642 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:15.885609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:15.932219 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:15.932174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:16.281544 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:16.281510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:16.488395 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:16.488363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:16.881508 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:16.881446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:17.681078 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:17.681049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:17.889460 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:17.889408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:18.086153 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:18.086119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:18.469647 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:18.469568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:19.533268 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:19.533230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:19.882053 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:19.881961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:20.083511 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:20.083469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:20.680600 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:20.680567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:21.290947 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:21.290907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:21.484698 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:21.484623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:21.683118 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:21.683028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:21.883477 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:21.883441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:22.480829 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.480790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:22.650332 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.650305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:22.650527 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.650340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:22.650868 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.650843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77208df7-adea-4cbd-88fb-9444a27a1afa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:22.652612 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.652594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77208df7-adea-4cbd-88fb-9444a27a1afa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5xjc\" (UID: \"77208df7-adea-4cbd-88fb-9444a27a1afa\") " pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:22.686479 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.686453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:22.755410 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.755361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5xjc"
Jul 19 14:51:22.874056 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:22.874023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5xjc"]
Jul 19 14:51:22.877624 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:51:22.877603    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77208df7_adea_4cbd_88fb_9444a27a1afa.slice/crio-23670748b086be81b12a6062d7e1b9fb69e22a8ac13cc861d096070a82f8a52d WatchSource:0}: Error finding container 23670748b086be81b12a6062d7e1b9fb69e22a8ac13cc861d096070a82f8a52d: Status 404 returned error can't find the container with id 23670748b086be81b12a6062d7e1b9fb69e22a8ac13cc861d096070a82f8a52d
Jul 19 14:51:23.583149 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:23.583062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5xjc" event={"ID":"77208df7-adea-4cbd-88fb-9444a27a1afa","Type":"ContainerStarted","Data":"23670748b086be81b12a6062d7e1b9fb69e22a8ac13cc861d096070a82f8a52d"}
Jul 19 14:51:24.047234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.047200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.151513 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.151461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.210198 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.210162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.346445 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.346355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.524674 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.524636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.578572 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.578537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.738193 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.738163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.914703 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.914663    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:24.965488 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:24.965459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:25.316304 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:25.316271    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:25.934042 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:25.934004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:26.131198 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:26.131161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:26.583489 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:26.583454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:26.754552 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:26.754519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:26.810218 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:26.810180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:28.602487 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:28.602413    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5xjc" event={"ID":"77208df7-adea-4cbd-88fb-9444a27a1afa","Type":"ContainerStarted","Data":"587dc3643c9f12861c8d2a270df2288aa0493910f96124248dc4a0c55ebe3a4e"}
Jul 19 14:51:28.936350 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:28.936261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:28.993183 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:28.993149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:30.462791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:30.462755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:30.559008 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:30.558971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:30.658356 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:30.658322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:31.614381 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:31.614342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5xjc" event={"ID":"77208df7-adea-4cbd-88fb-9444a27a1afa","Type":"ContainerStarted","Data":"a9c4322ae6ffc4481a8e65e6fbd00a20086e41bccbd143597b9289683e8c69aa"}
Jul 19 14:51:32.753184 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:32.753149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:32.806546 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:32.806518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:35.627371 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:35.627336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5xjc" event={"ID":"77208df7-adea-4cbd-88fb-9444a27a1afa","Type":"ContainerStarted","Data":"ee61d95436b8b7c2d6692e741d58a602ca6171479bf539ce8cef4d734d4d04a4"}
Jul 19 14:51:35.644850 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:35.644813    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t5xjc" podStartSLOduration=18.015330461 podStartE2EDuration="29.644800482s" totalImagesPullingTime="11.629470021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:51:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:35.64406475 +0000 UTC m=+276.669073824" watchObservedRunningTime="2026-07-19 14:51:35.644800482 +0000 UTC m=+276.669809555"
Jul 19 14:51:37.518599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:37.518562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:37.621036 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:37.621007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:37.684920 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:37.684894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:38.039557 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:38.039527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:40.280725 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:40.280676    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8rllc" podUID="7af39c6e-cb98-4b68-a6de-074e52edfd22"
Jul 19 14:51:40.641545 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:40.641453    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8rllc"
Jul 19 14:51:41.166979 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.166929    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:41.175072 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.175046    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6598b4494d-xn5dp"]
Jul 19 14:51:41.178013 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.177996    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.180465 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.180444    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 19 14:51:41.180465 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.180460    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 19 14:51:41.180637 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.180507    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 19 14:51:41.181362 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.181344    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7m7q9\""
Jul 19 14:51:41.181362 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.181354    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 19 14:51:41.181531 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.181486    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 19 14:51:41.181531 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.181484    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 19 14:51:41.181643 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.181528    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 19 14:51:41.188940 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.188921    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6598b4494d-xn5dp"]
Jul 19 14:51:41.292974 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.292936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr4cn\" (UniqueName: \"kubernetes.io/projected/b3c21bb0-cf69-471c-a794-938710850edb-kube-api-access-nr4cn\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.293367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.292979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-console-config\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.293367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.293003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-service-ca\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.293367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.293093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-serving-cert\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.293367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.293120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-oauth-config\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.293367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.293168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-oauth-serving-cert\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.394362 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.394323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-oauth-serving-cert\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.394560 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.394388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr4cn\" (UniqueName: \"kubernetes.io/projected/b3c21bb0-cf69-471c-a794-938710850edb-kube-api-access-nr4cn\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.394560 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.394414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-console-config\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.394560 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.394434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-service-ca\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.394560 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.394539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-serving-cert\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.394778 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.394573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-oauth-config\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.395128 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.395105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-service-ca\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.395128 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.395122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-oauth-serving-cert\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.395250 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.395191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-console-config\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.396870 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.396846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-oauth-config\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.396982 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.396969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-serving-cert\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.402249 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.402223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr4cn\" (UniqueName: \"kubernetes.io/projected/b3c21bb0-cf69-471c-a794-938710850edb-kube-api-access-nr4cn\") pod \"console-6598b4494d-xn5dp\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") " pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.486974 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.486926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:41.609097 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.609067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6598b4494d-xn5dp"]
Jul 19 14:51:41.612124 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:51:41.612101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c21bb0_cf69_471c_a794_938710850edb.slice/crio-c0350607dadecc3fd70d96c1161b9452c75b29c421ae1000ce5c5f888a1722e1 WatchSource:0}: Error finding container c0350607dadecc3fd70d96c1161b9452c75b29c421ae1000ce5c5f888a1722e1: Status 404 returned error can't find the container with id c0350607dadecc3fd70d96c1161b9452c75b29c421ae1000ce5c5f888a1722e1
Jul 19 14:51:41.645335 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.645307    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598b4494d-xn5dp" event={"ID":"b3c21bb0-cf69-471c-a794-938710850edb","Type":"ContainerStarted","Data":"c0350607dadecc3fd70d96c1161b9452c75b29c421ae1000ce5c5f888a1722e1"}
Jul 19 14:51:41.691580 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.691547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:41.749513 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.749426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:41.798081 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:41.798060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:42.358817 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.358784    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bdf48bfb5-w6lt6"]
Jul 19 14:51:42.361689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.361655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.362810 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.362784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:42.370040 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.370020    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 19 14:51:42.373348 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.373330    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bdf48bfb5-w6lt6"]
Jul 19 14:51:42.503874 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.503818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-trusted-ca-bundle\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.504052 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.503903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-oauth-serving-cert\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.504052 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.503941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-serving-cert\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.504052 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.503966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-oauth-config\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.504052 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.504044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45snm\" (UniqueName: \"kubernetes.io/projected/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-kube-api-access-45snm\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.504201 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.504140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-config\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.504239 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.504214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-service-ca\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.556573 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.556534    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:42.604562 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-trusted-ca-bundle\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.604724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-oauth-serving-cert\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.604724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-serving-cert\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.604724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-oauth-config\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.604724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45snm\" (UniqueName: \"kubernetes.io/projected/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-kube-api-access-45snm\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.604724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-config\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.604724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.604718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-service-ca\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.605422 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.605388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-config\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.605422 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.605403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-service-ca\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.605572 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.605385    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-oauth-serving-cert\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.605572 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.605563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-trusted-ca-bundle\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.607016 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.606998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-oauth-config\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.607130 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.607114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-serving-cert\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.612271 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.612226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45snm\" (UniqueName: \"kubernetes.io/projected/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-kube-api-access-45snm\") pod \"console-7bdf48bfb5-w6lt6\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") " pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.670889 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.670863    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:42.815392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:42.815312    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bdf48bfb5-w6lt6"]
Jul 19 14:51:42.817821 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:51:42.817796    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c87dd8_a0ae_498d_80f6_4d9f2b4dba5b.slice/crio-b2152a6390881dd4956071f52f236c92dd131b55aae097df39e8daadd9757758 WatchSource:0}: Error finding container b2152a6390881dd4956071f52f236c92dd131b55aae097df39e8daadd9757758: Status 404 returned error can't find the container with id b2152a6390881dd4956071f52f236c92dd131b55aae097df39e8daadd9757758
Jul 19 14:51:43.519873 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.519839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:43.571443 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.571410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:43.654453 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.654388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdf48bfb5-w6lt6" event={"ID":"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b","Type":"ContainerStarted","Data":"b2152a6390881dd4956071f52f236c92dd131b55aae097df39e8daadd9757758"}
Jul 19 14:51:43.784435 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.784360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:43.814689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.814655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:51:43.817506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.817467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7af39c6e-cb98-4b68-a6de-074e52edfd22-metrics-tls\") pod \"dns-default-8rllc\" (UID: \"7af39c6e-cb98-4b68-a6de-074e52edfd22\") " pod="openshift-dns/dns-default-8rllc"
Jul 19 14:51:43.835800 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.835774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:43.944603 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.944575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4r258\""
Jul 19 14:51:43.952656 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:43.952624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8rllc"
Jul 19 14:51:44.759616 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:44.759573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:44.819738 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:44.819706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:44.984654 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:44.984633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:45.040038 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.040016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8rllc"]
Jul 19 14:51:45.042351 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:51:45.042320    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af39c6e_cb98_4b68_a6de_074e52edfd22.slice/crio-582ace43b3858379c06468922a650109a08fa7359810d87c58debcd0bf8b0ea8 WatchSource:0}: Error finding container 582ace43b3858379c06468922a650109a08fa7359810d87c58debcd0bf8b0ea8: Status 404 returned error can't find the container with id 582ace43b3858379c06468922a650109a08fa7359810d87c58debcd0bf8b0ea8
Jul 19 14:51:45.048025 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.048002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:45.664222 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.664183    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdf48bfb5-w6lt6" event={"ID":"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b","Type":"ContainerStarted","Data":"45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f"}
Jul 19 14:51:45.665857 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.665823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598b4494d-xn5dp" event={"ID":"b3c21bb0-cf69-471c-a794-938710850edb","Type":"ContainerStarted","Data":"e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d"}
Jul 19 14:51:45.666895 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.666872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8rllc" event={"ID":"7af39c6e-cb98-4b68-a6de-074e52edfd22","Type":"ContainerStarted","Data":"582ace43b3858379c06468922a650109a08fa7359810d87c58debcd0bf8b0ea8"}
Jul 19 14:51:45.680400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.680350    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bdf48bfb5-w6lt6" podStartSLOduration=1.5046792450000002 podStartE2EDuration="3.680333918s" totalImagesPullingTime="2.175654673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:45.67979925 +0000 UTC m=+286.704808326" watchObservedRunningTime="2026-07-19 14:51:45.680333918 +0000 UTC m=+286.705342992"
Jul 19 14:51:45.696367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:45.696326    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6598b4494d-xn5dp" podStartSLOduration=1.323340725 podStartE2EDuration="4.696313882s" totalImagesPullingTime="3.372973157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:45.694839238 +0000 UTC m=+286.719848314" watchObservedRunningTime="2026-07-19 14:51:45.696313882 +0000 UTC m=+286.721323001"
Jul 19 14:51:46.160634 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:46.160586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:46.209838 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:46.209809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:46.672463 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:46.672431    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8rllc" event={"ID":"7af39c6e-cb98-4b68-a6de-074e52edfd22","Type":"ContainerStarted","Data":"9c09285e885751bb2434fd0a4a6e4ad96b7e67cac2ba8d8cf4ccca0138d8f559"}
Jul 19 14:51:47.676962 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:47.676927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8rllc" event={"ID":"7af39c6e-cb98-4b68-a6de-074e52edfd22","Type":"ContainerStarted","Data":"37bc863525d09ebb98ea68686856564d0588b55d22215a06395487ab9a114105"}
Jul 19 14:51:47.677325 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:47.677048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8rllc"
Jul 19 14:51:47.699817 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:47.699778    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8rllc" podStartSLOduration=253.150195932 podStartE2EDuration="4m14.69976456s" totalImagesPullingTime="1.549568628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:47.69824894 +0000 UTC m=+288.723258015" watchObservedRunningTime="2026-07-19 14:51:47.69976456 +0000 UTC m=+288.724773636"
Jul 19 14:51:47.795589 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:47.795558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:47.899223 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:47.899193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:49.553305 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:49.553276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:49.647168 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:49.647138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:50.616881 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:50.616853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:51.128001 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.127954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:51.228892 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.228862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:51.293824 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.293795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:51.487998 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.487973    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:51.488203 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.488055    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:51.492815 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.492795    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:51.695079 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.695054    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:51:51.832603 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.832525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:51.936808 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:51.936778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:52.599234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.599177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:52.671961 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.671936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:52.672101 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.671979    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:52.676429 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.676410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:52.698297 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.698273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:52.698860 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.698844    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:51:52.740883 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.740857    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6598b4494d-xn5dp"]
Jul 19 14:51:52.839354 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.839319    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:52.940447 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:52.940357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:53.074577 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.074543    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:53.156287 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.156260    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jvqhz"]
Jul 19 14:51:53.159275 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.159256    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.163506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163471    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 19 14:51:53.163506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163488    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tfpqx\""
Jul 19 14:51:53.163810 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163788    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 19 14:51:53.163810 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163804    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 19 14:51:53.163997 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163842    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 19 14:51:53.163997 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 19 14:51:53.163997 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.163953    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 19 14:51:53.177113 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.177092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:53.191820 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.191762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9vf\" (UniqueName: \"kubernetes.io/projected/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-kube-api-access-lx9vf\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.191820 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.191805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-textfile\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.191958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.191866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-metrics-client-ca\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.191958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.191922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-root\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.191958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.191949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-tls\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.192107 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.191982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-sys\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.192107 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.192017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-wtmp\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.192107 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.192054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.192264 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.192119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293343 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9vf\" (UniqueName: \"kubernetes.io/projected/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-kube-api-access-lx9vf\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293462 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-textfile\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293462 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-metrics-client-ca\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293462 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-root\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293462 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-root\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-tls\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-sys\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-wtmp\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293923 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-wtmp\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293923 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-textfile\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.293923 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.293890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-sys\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.294170 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.294150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-metrics-client-ca\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.294232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.294187    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.295847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.295828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.296244 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.296226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-node-exporter-tls\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.303333 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.303304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9vf\" (UniqueName: \"kubernetes.io/projected/0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd-kube-api-access-lx9vf\") pod \"node-exporter-jvqhz\" (UID: \"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd\") " pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.305537 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.305481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:53.405434 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.405401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:53.469539 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.469461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jvqhz"
Jul 19 14:51:53.480889 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:51:53.480863    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e13dd8a_9f3d_4479_a4cb_0e2c4c28f6bd.slice/crio-61ccdd3a48f89ac12c494d838263a253e1a3db94d5eb08d3979dc593cbd6c865 WatchSource:0}: Error finding container 61ccdd3a48f89ac12c494d838263a253e1a3db94d5eb08d3979dc593cbd6c865: Status 404 returned error can't find the container with id 61ccdd3a48f89ac12c494d838263a253e1a3db94d5eb08d3979dc593cbd6c865
Jul 19 14:51:53.505215 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.505191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:53.534312 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:51:53.534284    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pdcdf" podUID="89704d43-d416-4b1b-853c-52ecefa8b7a1"
Jul 19 14:51:53.700388 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.700354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvqhz" event={"ID":"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd","Type":"ContainerStarted","Data":"61ccdd3a48f89ac12c494d838263a253e1a3db94d5eb08d3979dc593cbd6c865"}
Jul 19 14:51:53.885982 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:53.885949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.069177 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.069147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.149549 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.149465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.290866 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.290837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.390453 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.390422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.514063 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.514032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.704880 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.704853    2577 generic.go:356] "Generic (PLEG): container finished" podID="0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd" containerID="5b642b19261d341035a24dd46b76335725dd116806a73dcf386eacb846e7f842" exitCode=0
Jul 19 14:51:54.705171 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.704889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvqhz" event={"ID":"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd","Type":"ContainerDied","Data":"5b642b19261d341035a24dd46b76335725dd116806a73dcf386eacb846e7f842"}
Jul 19 14:51:54.875187 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.875143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:54.982020 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:54.981993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:55.710455 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:55.710420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvqhz" event={"ID":"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd","Type":"ContainerStarted","Data":"cdb86594412fd3e084e7f5d36afcc13d4bb9acf16d4fdfd79246a77a0d2c2b33"}
Jul 19 14:51:55.710935 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:55.710476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvqhz" event={"ID":"0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd","Type":"ContainerStarted","Data":"22e93ec7acbe4a2d899842098037aae6be72ca6f54b3d727543a829cd11a812c"}
Jul 19 14:51:55.728442 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:55.728398    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jvqhz" podStartSLOduration=1.642176729 podStartE2EDuration="2.728384244s" totalImagesPullingTime="1.022046428s" totalInitContainerRuntime="64.161087ms" isStatefulPod=true podCreationTimestamp="2026-07-19 14:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:55.727333401 +0000 UTC m=+296.752342513" watchObservedRunningTime="2026-07-19 14:51:55.728384244 +0000 UTC m=+296.753393317"
Jul 19 14:51:56.025936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:56.025904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:56.129644 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:56.129615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:56.486313 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:56.486282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:56.587078 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:56.587051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:57.682841 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:57.682814    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8rllc"
Jul 19 14:51:57.875783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:57.875754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:57.937675 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:57.937603    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:58.048749 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:58.048716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:58.149151 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:58.149125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:58.941228 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:58.941199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:59.041971 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:59.041938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:59.308141 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:59.308099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:59.352459 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:51:59.352440    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 19 14:52:04.454891 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:04.454858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:05.533964 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:05.533933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:52:06.124940 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:06.124910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:06.184138 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:06.184108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:06.880789 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:06.880757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:06.981659 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:06.981634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:08.280872 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:08.280842    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bdf48bfb5-w6lt6"]
Jul 19 14:52:08.446978 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:08.446935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:08.547238 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:08.547141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:13.937574 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:13.937536    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:13.990469 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:13.990444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:17.788630 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:17.788572    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" podUID="e5cc234a-b64a-449e-8c43-4279feb986dd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 19 14:52:18.719368 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:18.719305    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6598b4494d-xn5dp" podUID="b3c21bb0-cf69-471c-a794-938710850edb" containerName="console" containerID="cri-o://e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d" gracePeriod=15
Jul 19 14:52:18.948792 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:18.948766    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:52:19.099716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.099680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-console-config\") pod \"b3c21bb0-cf69-471c-a794-938710850edb\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") "
Jul 19 14:52:19.099716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.099725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-service-ca\") pod \"b3c21bb0-cf69-471c-a794-938710850edb\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") "
Jul 19 14:52:19.099933 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.099745    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-oauth-config\") pod \"b3c21bb0-cf69-471c-a794-938710850edb\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") "
Jul 19 14:52:19.099933 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.099770    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr4cn\" (UniqueName: \"kubernetes.io/projected/b3c21bb0-cf69-471c-a794-938710850edb-kube-api-access-nr4cn\") pod \"b3c21bb0-cf69-471c-a794-938710850edb\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") "
Jul 19 14:52:19.099933 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.099803    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-serving-cert\") pod \"b3c21bb0-cf69-471c-a794-938710850edb\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") "
Jul 19 14:52:19.099933 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.099829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-oauth-serving-cert\") pod \"b3c21bb0-cf69-471c-a794-938710850edb\" (UID: \"b3c21bb0-cf69-471c-a794-938710850edb\") "
Jul 19 14:52:19.100167 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.100140    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-console-config" (OuterVolumeSpecName: "console-config") pod "b3c21bb0-cf69-471c-a794-938710850edb" (UID: "b3c21bb0-cf69-471c-a794-938710850edb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:19.100242 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.100222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b3c21bb0-cf69-471c-a794-938710850edb" (UID: "b3c21bb0-cf69-471c-a794-938710850edb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:19.100289 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.100244    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-service-ca" (OuterVolumeSpecName: "service-ca") pod "b3c21bb0-cf69-471c-a794-938710850edb" (UID: "b3c21bb0-cf69-471c-a794-938710850edb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:19.101965 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.101919    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3c21bb0-cf69-471c-a794-938710850edb-kube-api-access-nr4cn" (OuterVolumeSpecName: "kube-api-access-nr4cn") pod "b3c21bb0-cf69-471c-a794-938710850edb" (UID: "b3c21bb0-cf69-471c-a794-938710850edb"). InnerVolumeSpecName "kube-api-access-nr4cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:52:19.102140 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.102123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b3c21bb0-cf69-471c-a794-938710850edb" (UID: "b3c21bb0-cf69-471c-a794-938710850edb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:52:19.102191 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.102150    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b3c21bb0-cf69-471c-a794-938710850edb" (UID: "b3c21bb0-cf69-471c-a794-938710850edb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:52:19.200739 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.200715    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:19.200739 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.200735    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-oauth-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:19.200873 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.200746    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-console-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:19.200873 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.200754    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3c21bb0-cf69-471c-a794-938710850edb-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:19.200873 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.200761    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3c21bb0-cf69-471c-a794-938710850edb-console-oauth-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:19.200873 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.200770    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr4cn\" (UniqueName: \"kubernetes.io/projected/b3c21bb0-cf69-471c-a794-938710850edb-kube-api-access-nr4cn\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:19.787845 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.787811    2577 generic.go:356] "Generic (PLEG): container finished" podID="b3c21bb0-cf69-471c-a794-938710850edb" containerID="e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d" exitCode=0
Jul 19 14:52:19.788022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.787855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598b4494d-xn5dp" event={"ID":"b3c21bb0-cf69-471c-a794-938710850edb","Type":"ContainerDied","Data":"e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d"}
Jul 19 14:52:19.788022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.787872    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598b4494d-xn5dp"
Jul 19 14:52:19.788022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.787919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598b4494d-xn5dp" event={"ID":"b3c21bb0-cf69-471c-a794-938710850edb","Type":"ContainerDied","Data":"c0350607dadecc3fd70d96c1161b9452c75b29c421ae1000ce5c5f888a1722e1"}
Jul 19 14:52:19.788022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.787939    2577 scope.go:117] "RemoveContainer" containerID="e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d"
Jul 19 14:52:19.795594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.795577    2577 scope.go:117] "RemoveContainer" containerID="e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d"
Jul 19 14:52:19.795845 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:52:19.795824    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d\": container with ID starting with e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d not found: ID does not exist" containerID="e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d"
Jul 19 14:52:19.795916 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.795856    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d"} err="failed to get container status \"e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d\": rpc error: code = NotFound desc = could not find container \"e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d\": container with ID starting with e6cf400fd12c8746226eb705c2cb13da05dea5b6f9275ddc4a651a6dd549d86d not found: ID does not exist"
Jul 19 14:52:19.803523 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.803455    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6598b4494d-xn5dp"]
Jul 19 14:52:19.806837 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:19.806817    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6598b4494d-xn5dp"]
Jul 19 14:52:21.538061 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:21.538031    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3c21bb0-cf69-471c-a794-938710850edb" path="/var/lib/kubelet/pods/b3c21bb0-cf69-471c-a794-938710850edb/volumes"
Jul 19 14:52:27.788056 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:27.788018    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" podUID="e5cc234a-b64a-449e-8c43-4279feb986dd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 19 14:52:28.940191 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:28.940135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:29.037474 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:29.037444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:33.304850 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.304808    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bdf48bfb5-w6lt6" podUID="32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" containerName="console" containerID="cri-o://45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f" gracePeriod=15
Jul 19 14:52:33.536715 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.536692    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:52:33.703990 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.703918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45snm\" (UniqueName: \"kubernetes.io/projected/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-kube-api-access-45snm\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.703990 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.703948    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-trusted-ca-bundle\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.703990 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.703972    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-oauth-config\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.704245 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-service-ca\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.704245 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-config\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.704245 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-serving-cert\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.704245 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704085    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-oauth-serving-cert\") pod \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\" (UID: \"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b\") "
Jul 19 14:52:33.704532 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704483    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-service-ca" (OuterVolumeSpecName: "service-ca") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:33.704610 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704526    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:33.704610 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-config" (OuterVolumeSpecName: "console-config") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:33.704610 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.704560    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:52:33.706135 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.706109    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:52:33.706234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.706143    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:52:33.706234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.706177    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-kube-api-access-45snm" (OuterVolumeSpecName: "kube-api-access-45snm") pod "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" (UID: "32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b"). InnerVolumeSpecName "kube-api-access-45snm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:52:33.804863 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804835    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.804863 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804858    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.804863 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804867    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.805033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804876    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-oauth-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.805033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804884    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45snm\" (UniqueName: \"kubernetes.io/projected/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-kube-api-access-45snm\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.805033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804893    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-trusted-ca-bundle\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.805033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.804905    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b-console-oauth-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:52:33.831180 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.831151    2577 generic.go:356] "Generic (PLEG): container finished" podID="32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" containerID="45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f" exitCode=0
Jul 19 14:52:33.831270 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.831193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdf48bfb5-w6lt6"
Jul 19 14:52:33.831270 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.831238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdf48bfb5-w6lt6" event={"ID":"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b","Type":"ContainerDied","Data":"45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f"}
Jul 19 14:52:33.831360 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.831288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdf48bfb5-w6lt6" event={"ID":"32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b","Type":"ContainerDied","Data":"b2152a6390881dd4956071f52f236c92dd131b55aae097df39e8daadd9757758"}
Jul 19 14:52:33.831360 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.831307    2577 scope.go:117] "RemoveContainer" containerID="45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f"
Jul 19 14:52:33.838812 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.838700    2577 scope.go:117] "RemoveContainer" containerID="45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f"
Jul 19 14:52:33.839110 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:52:33.839086    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f\": container with ID starting with 45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f not found: ID does not exist" containerID="45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f"
Jul 19 14:52:33.839179 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.839117    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f"} err="failed to get container status \"45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f\": rpc error: code = NotFound desc = could not find container \"45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f\": container with ID starting with 45d7b9399676acf3c4ec7620016230ca4b56b4087e68b0c58907ba388c7c719f not found: ID does not exist"
Jul 19 14:52:33.852412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.852391    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bdf48bfb5-w6lt6"]
Jul 19 14:52:33.858184 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:33.858167    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bdf48bfb5-w6lt6"]
Jul 19 14:52:35.537594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:35.537561    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" path="/var/lib/kubelet/pods/32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b/volumes"
Jul 19 14:52:37.788568 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:37.788533    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" podUID="e5cc234a-b64a-449e-8c43-4279feb986dd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 19 14:52:37.788913 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:37.788597    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp"
Jul 19 14:52:37.789104 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:37.789085    2577 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ae326e061b2ad30c4f8d3fc865cc3d7a7a0e42aad0e65dbd94550e4b7fda5def"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 19 14:52:37.789144 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:37.789128    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" podUID="e5cc234a-b64a-449e-8c43-4279feb986dd" containerName="service-proxy" containerID="cri-o://ae326e061b2ad30c4f8d3fc865cc3d7a7a0e42aad0e65dbd94550e4b7fda5def" gracePeriod=30
Jul 19 14:52:38.849802 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:38.849771    2577 generic.go:356] "Generic (PLEG): container finished" podID="e5cc234a-b64a-449e-8c43-4279feb986dd" containerID="ae326e061b2ad30c4f8d3fc865cc3d7a7a0e42aad0e65dbd94550e4b7fda5def" exitCode=2
Jul 19 14:52:38.850174 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:38.849831    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" event={"ID":"e5cc234a-b64a-449e-8c43-4279feb986dd","Type":"ContainerDied","Data":"ae326e061b2ad30c4f8d3fc865cc3d7a7a0e42aad0e65dbd94550e4b7fda5def"}
Jul 19 14:52:38.850174 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:38.849862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57d48659d6-2chjp" event={"ID":"e5cc234a-b64a-449e-8c43-4279feb986dd","Type":"ContainerStarted","Data":"b7529195ec3371f47c486bc1f4b4fd35a08f56fa1d6bcb95e96211e085ddd6dc"}
Jul 19 14:52:52.148324 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:52.148241    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:52.201645 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:52.201600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:54.170068 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:54.170039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:54.224009 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:52:54.223983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:12.301692 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.301658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:12.402423 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.402392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:12.452846 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.452814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:53:12.455064 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.455046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89704d43-d416-4b1b-853c-52ecefa8b7a1-metrics-certs\") pod \"network-metrics-daemon-pdcdf\" (UID: \"89704d43-d416-4b1b-853c-52ecefa8b7a1\") " pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:53:12.738091 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.738071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5p5rq\""
Jul 19 14:53:12.745119 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.745100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pdcdf"
Jul 19 14:53:12.868295 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.868267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pdcdf"]
Jul 19 14:53:12.870694 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:53:12.870659    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89704d43_d416_4b1b_853c_52ecefa8b7a1.slice/crio-d83a4c65a6cdc71b692e2c04d425beffb5fc296ba79727ab35aa8746ce1bbbb1 WatchSource:0}: Error finding container d83a4c65a6cdc71b692e2c04d425beffb5fc296ba79727ab35aa8746ce1bbbb1: Status 404 returned error can't find the container with id d83a4c65a6cdc71b692e2c04d425beffb5fc296ba79727ab35aa8746ce1bbbb1
Jul 19 14:53:12.872718 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.872702    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:53:12.964379 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:12.964346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pdcdf" event={"ID":"89704d43-d416-4b1b-853c-52ecefa8b7a1","Type":"ContainerStarted","Data":"d83a4c65a6cdc71b692e2c04d425beffb5fc296ba79727ab35aa8746ce1bbbb1"}
Jul 19 14:53:14.973763 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:14.973724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pdcdf" event={"ID":"89704d43-d416-4b1b-853c-52ecefa8b7a1","Type":"ContainerStarted","Data":"1ddc10f109c856d6314ef44d73559e4cdfe104932f342ad0b5ebe6b0254375b0"}
Jul 19 14:53:14.974135 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:14.973778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pdcdf" event={"ID":"89704d43-d416-4b1b-853c-52ecefa8b7a1","Type":"ContainerStarted","Data":"3cf61ea2d82b14249db2837a7c5fed231552fbf7c54b6d42bbdf28db3b49a0d8"}
Jul 19 14:53:14.994792 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:14.994749    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pdcdf" podStartSLOduration=374.966651755 podStartE2EDuration="6m15.994734301s" totalImagesPullingTime="1.028082546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:53:14.993833264 +0000 UTC m=+376.018842341" watchObservedRunningTime="2026-07-19 14:53:14.994734301 +0000 UTC m=+376.019743374"
Jul 19 14:53:15.079756 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:15.079721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:15.135835 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:15.135807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:17.381307 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381278    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c745b67b9-gfj5n"]
Jul 19 14:53:17.381689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381482    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" containerName="console"
Jul 19 14:53:17.381689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381507    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" containerName="console"
Jul 19 14:53:17.381689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381520    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3c21bb0-cf69-471c-a794-938710850edb" containerName="console"
Jul 19 14:53:17.381689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381525    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3c21bb0-cf69-471c-a794-938710850edb" containerName="console"
Jul 19 14:53:17.381689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381569    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3c21bb0-cf69-471c-a794-938710850edb" containerName="console"
Jul 19 14:53:17.381689 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.381577    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c87dd8-a0ae-498d-80f6-4d9f2b4dba5b" containerName="console"
Jul 19 14:53:17.385834 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.385807    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.388533 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.388510    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kk5pn\""
Jul 19 14:53:17.388757 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.388742    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 19 14:53:17.388960 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.388942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 19 14:53:17.389028 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.388942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 19 14:53:17.389028 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.389002    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 19 14:53:17.389445 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.389426    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 19 14:53:17.394478 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.394425    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 19 14:53:17.396545 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.396523    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c745b67b9-gfj5n"]
Jul 19 14:53:17.486594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-telemeter-client-tls\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-secret-telemeter-client\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-metrics-client-ca\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxgt\" (UniqueName: \"kubernetes.io/projected/81121ffb-4312-437a-8eab-ac80374e7c8a-kube-api-access-rfxgt\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486842 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-federate-client-tls\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486842 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.486842 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.486787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-serving-certs-ca-bundle\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587212 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-telemeter-client-tls\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587354 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-secret-telemeter-client\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587354 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-metrics-client-ca\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587354 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxgt\" (UniqueName: \"kubernetes.io/projected/81121ffb-4312-437a-8eab-ac80374e7c8a-kube-api-access-rfxgt\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587354 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-federate-client-tls\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.587594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.587417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-serving-certs-ca-bundle\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.588085 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.588063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-metrics-client-ca\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.588204 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.588103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-serving-certs-ca-bundle\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.588739 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.588715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81121ffb-4312-437a-8eab-ac80374e7c8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.589950 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.589927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-federate-client-tls\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.590019 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.589997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.590194 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.590178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-telemeter-client-tls\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.590234 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.590204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/81121ffb-4312-437a-8eab-ac80374e7c8a-secret-telemeter-client\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.594879 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.594858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxgt\" (UniqueName: \"kubernetes.io/projected/81121ffb-4312-437a-8eab-ac80374e7c8a-kube-api-access-rfxgt\") pod \"telemeter-client-c745b67b9-gfj5n\" (UID: \"81121ffb-4312-437a-8eab-ac80374e7c8a\") " pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.695928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.695864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n"
Jul 19 14:53:17.822242 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.822214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c745b67b9-gfj5n"]
Jul 19 14:53:17.825438 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:53:17.825414    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81121ffb_4312_437a_8eab_ac80374e7c8a.slice/crio-24a8abda36656ffadd8eff7a7d45fed5736425edfb5e14ec49ddd3ce18ffe54f WatchSource:0}: Error finding container 24a8abda36656ffadd8eff7a7d45fed5736425edfb5e14ec49ddd3ce18ffe54f: Status 404 returned error can't find the container with id 24a8abda36656ffadd8eff7a7d45fed5736425edfb5e14ec49ddd3ce18ffe54f
Jul 19 14:53:17.986003 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:17.985971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n" event={"ID":"81121ffb-4312-437a-8eab-ac80374e7c8a","Type":"ContainerStarted","Data":"24a8abda36656ffadd8eff7a7d45fed5736425edfb5e14ec49ddd3ce18ffe54f"}
Jul 19 14:53:19.994409 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:19.994375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n" event={"ID":"81121ffb-4312-437a-8eab-ac80374e7c8a","Type":"ContainerStarted","Data":"5bb225d32b9a2c9b83846f57f7e97e0bbaf8f0dc4dce12460067b341804d669d"}
Jul 19 14:53:21.000583 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.000553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n" event={"ID":"81121ffb-4312-437a-8eab-ac80374e7c8a","Type":"ContainerStarted","Data":"5e88253141508a6e20bc169887086b7d5b2ef1de85a674d2c026cb67800f2fc9"}
Jul 19 14:53:21.000878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.000641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n" event={"ID":"81121ffb-4312-437a-8eab-ac80374e7c8a","Type":"ContainerStarted","Data":"e4b73f279cee483b127ac423ea0eb3cdbfdb15c8b574c378683d9883ce623843"}
Jul 19 14:53:21.022376 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.022333    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c745b67b9-gfj5n" podStartSLOduration=1.041036634 podStartE2EDuration="4.02231865s" totalImagesPullingTime="2.981282016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:53:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:53:21.020318566 +0000 UTC m=+382.045327641" watchObservedRunningTime="2026-07-19 14:53:21.02231865 +0000 UTC m=+382.047327720"
Jul 19 14:53:21.752895 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.752866    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fb96885cf-qmjp2"]
Jul 19 14:53:21.755773 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.755757    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.758847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.758826    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 19 14:53:21.759011 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.758849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 19 14:53:21.759011 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.758991    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7m7q9\""
Jul 19 14:53:21.759011 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.758996    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 19 14:53:21.759178 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.759063    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 19 14:53:21.759220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.759204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 19 14:53:21.759294 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.759277    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 19 14:53:21.759344 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.759311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 19 14:53:21.763225 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.763205    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 19 14:53:21.767128 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.767108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fb96885cf-qmjp2"]
Jul 19 14:53:21.819059 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-oauth-config\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.819194 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819063    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-serving-cert\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.819194 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-console-config\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.819194 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-service-ca\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.819361 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-oauth-serving-cert\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.819361 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-trusted-ca-bundle\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.819361 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.819255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr57t\" (UniqueName: \"kubernetes.io/projected/525501f0-f14f-4212-9f50-f47957ba0e35-kube-api-access-pr57t\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.829775 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.829748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:21.919908 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.919877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-serving-cert\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.919913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-console-config\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.919934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-service-ca\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.919960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-oauth-serving-cert\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.919981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-trusted-ca-bundle\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.920017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr57t\" (UniqueName: \"kubernetes.io/projected/525501f0-f14f-4212-9f50-f47957ba0e35-kube-api-access-pr57t\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.920071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-oauth-config\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920798 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.920770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-console-config\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920883 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.920788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-oauth-serving-cert\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.920883 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.920830    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-service-ca\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.921278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.921258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-trusted-ca-bundle\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.922593 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.922568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-oauth-config\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.922662 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.922601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-serving-cert\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.930131 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.930107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr57t\" (UniqueName: \"kubernetes.io/projected/525501f0-f14f-4212-9f50-f47957ba0e35-kube-api-access-pr57t\") pod \"console-7fb96885cf-qmjp2\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") " pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:21.931415 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.931394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:21.981526 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:21.981485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:22.065280 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:22.065196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:22.186067 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:22.186045    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fb96885cf-qmjp2"]
Jul 19 14:53:22.188356 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:53:22.188327    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525501f0_f14f_4212_9f50_f47957ba0e35.slice/crio-fc7156eec965cba3f61382ed4150d444c8c7ea5726c91605cb2cdfa8af3da154 WatchSource:0}: Error finding container fc7156eec965cba3f61382ed4150d444c8c7ea5726c91605cb2cdfa8af3da154: Status 404 returned error can't find the container with id fc7156eec965cba3f61382ed4150d444c8c7ea5726c91605cb2cdfa8af3da154
Jul 19 14:53:22.551802 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:22.551769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:22.600712 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:22.600685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:23.008642 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:23.008607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb96885cf-qmjp2" event={"ID":"525501f0-f14f-4212-9f50-f47957ba0e35","Type":"ContainerStarted","Data":"82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57"}
Jul 19 14:53:23.008820 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:23.008654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb96885cf-qmjp2" event={"ID":"525501f0-f14f-4212-9f50-f47957ba0e35","Type":"ContainerStarted","Data":"fc7156eec965cba3f61382ed4150d444c8c7ea5726c91605cb2cdfa8af3da154"}
Jul 19 14:53:23.026174 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:23.026124    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fb96885cf-qmjp2" podStartSLOduration=2.026111167 podStartE2EDuration="2.026111167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:53:23.024629978 +0000 UTC m=+384.049639069" watchObservedRunningTime="2026-07-19 14:53:23.026111167 +0000 UTC m=+384.051120240"
Jul 19 14:53:24.100075 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:24.100047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:32.065505 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:32.065453    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:32.065974 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:32.065528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:32.070173 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:32.070153    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:33.048627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:33.048601    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:53:33.194294 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:33.194263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:33.256786 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:33.256759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:33.382185 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:33.382116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:53:33.429610 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:53:33.429590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:03.160046 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:03.160013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:03.216730 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:03.216703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:27.544382 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:27.544296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:27.594758 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:27.594722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:35.251026 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.250992    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-579bf5847f-bljhq"]
Jul 19 14:54:35.253841 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.253823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.265330 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.265307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-579bf5847f-bljhq"]
Jul 19 14:54:35.305559 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.305530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:35.321825 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.321801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-service-ca\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.321897 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.321836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-trusted-ca-bundle\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.321897 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.321864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-oauth-config\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.321897 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.321880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-serving-cert\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.321992 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.321921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-config\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.321992 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.321977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg8sd\" (UniqueName: \"kubernetes.io/projected/70308b9b-72bb-4fe1-8833-505cdfa99e1b-kube-api-access-hg8sd\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.322053 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.322003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-oauth-serving-cert\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.356757 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.356729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:35.422633 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg8sd\" (UniqueName: \"kubernetes.io/projected/70308b9b-72bb-4fe1-8833-505cdfa99e1b-kube-api-access-hg8sd\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.422732 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-oauth-serving-cert\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.422732 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-service-ca\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.422732 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-trusted-ca-bundle\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.422892 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-oauth-config\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.422892 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-serving-cert\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.423024 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.422963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-config\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.423437 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.423408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-oauth-serving-cert\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.423571 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.423545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-trusted-ca-bundle\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.423637 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.423575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-config\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.423695 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.423641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-service-ca\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.425184 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.425167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-oauth-config\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.425293 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.425276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-serving-cert\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.430238 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.430217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg8sd\" (UniqueName: \"kubernetes.io/projected/70308b9b-72bb-4fe1-8833-505cdfa99e1b-kube-api-access-hg8sd\") pod \"console-579bf5847f-bljhq\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") " pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.562673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.562605    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:35.685051 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:35.685023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-579bf5847f-bljhq"]
Jul 19 14:54:35.688226 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:54:35.688180    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70308b9b_72bb_4fe1_8833_505cdfa99e1b.slice/crio-e993c19866de5302fe49e97cd92855826759e46c4ba23392438b4956255e2848 WatchSource:0}: Error finding container e993c19866de5302fe49e97cd92855826759e46c4ba23392438b4956255e2848: Status 404 returned error can't find the container with id e993c19866de5302fe49e97cd92855826759e46c4ba23392438b4956255e2848
Jul 19 14:54:36.254711 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:36.254679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579bf5847f-bljhq" event={"ID":"70308b9b-72bb-4fe1-8833-505cdfa99e1b","Type":"ContainerStarted","Data":"197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3"}
Jul 19 14:54:36.255107 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:36.254727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579bf5847f-bljhq" event={"ID":"70308b9b-72bb-4fe1-8833-505cdfa99e1b","Type":"ContainerStarted","Data":"e993c19866de5302fe49e97cd92855826759e46c4ba23392438b4956255e2848"}
Jul 19 14:54:36.275121 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:36.275070    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-579bf5847f-bljhq" podStartSLOduration=1.275054242 podStartE2EDuration="1.275054242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:54:36.273662599 +0000 UTC m=+457.298671673" watchObservedRunningTime="2026-07-19 14:54:36.275054242 +0000 UTC m=+457.300063316"
Jul 19 14:54:39.437562 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:39.437524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:39.539228 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:39.539191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:43.945650 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:43.945618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:44.045936 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:44.045900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:45.562724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:45.562696    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:45.563161 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:45.562981    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:45.567236 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:45.567220    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:46.291393 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:46.291366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:54:46.333929 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:46.333899    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fb96885cf-qmjp2"]
Jul 19 14:54:46.436049 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:46.436022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:46.536531 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:54:46.536481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:11.353702 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.353638    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fb96885cf-qmjp2" podUID="525501f0-f14f-4212-9f50-f47957ba0e35" containerName="console" containerID="cri-o://82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57" gracePeriod=15
Jul 19 14:55:11.598251 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.598229    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:55:11.676386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676303    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr57t\" (UniqueName: \"kubernetes.io/projected/525501f0-f14f-4212-9f50-f47957ba0e35-kube-api-access-pr57t\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676343    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-serving-cert\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676374    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-trusted-ca-bundle\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676684 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676428    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-console-config\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676684 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676468    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-oauth-serving-cert\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676684 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676610    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-service-ca\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676684 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676659    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-oauth-config\") pod \"525501f0-f14f-4212-9f50-f47957ba0e35\" (UID: \"525501f0-f14f-4212-9f50-f47957ba0e35\") "
Jul 19 14:55:11.676916 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676889    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-console-config" (OuterVolumeSpecName: "console-config") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:55:11.676916 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676903    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:55:11.677022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:55:11.677022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.676932    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-service-ca" (OuterVolumeSpecName: "service-ca") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:55:11.678536 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.678477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:55:11.678648 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.678550    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525501f0-f14f-4212-9f50-f47957ba0e35-kube-api-access-pr57t" (OuterVolumeSpecName: "kube-api-access-pr57t") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "kube-api-access-pr57t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:11.678706 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.678646    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "525501f0-f14f-4212-9f50-f47957ba0e35" (UID: "525501f0-f14f-4212-9f50-f47957ba0e35"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:55:11.778120 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778081    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr57t\" (UniqueName: \"kubernetes.io/projected/525501f0-f14f-4212-9f50-f47957ba0e35-kube-api-access-pr57t\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:11.778120 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778111    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:11.778120 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778121    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-trusted-ca-bundle\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:11.778120 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778129    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-console-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:11.778386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778138    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-oauth-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:11.778386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778147    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/525501f0-f14f-4212-9f50-f47957ba0e35-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:11.778386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:11.778155    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/525501f0-f14f-4212-9f50-f47957ba0e35-console-oauth-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:55:12.375209 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.375179    2577 generic.go:356] "Generic (PLEG): container finished" podID="525501f0-f14f-4212-9f50-f47957ba0e35" containerID="82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57" exitCode=0
Jul 19 14:55:12.375652 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.375220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb96885cf-qmjp2" event={"ID":"525501f0-f14f-4212-9f50-f47957ba0e35","Type":"ContainerDied","Data":"82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57"}
Jul 19 14:55:12.375652 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.375221    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb96885cf-qmjp2"
Jul 19 14:55:12.375652 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.375268    2577 scope.go:117] "RemoveContainer" containerID="82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57"
Jul 19 14:55:12.375652 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.375257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb96885cf-qmjp2" event={"ID":"525501f0-f14f-4212-9f50-f47957ba0e35","Type":"ContainerDied","Data":"fc7156eec965cba3f61382ed4150d444c8c7ea5726c91605cb2cdfa8af3da154"}
Jul 19 14:55:12.383818 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.383797    2577 scope.go:117] "RemoveContainer" containerID="82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57"
Jul 19 14:55:12.384085 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:55:12.384063    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57\": container with ID starting with 82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57 not found: ID does not exist" containerID="82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57"
Jul 19 14:55:12.384153 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.384094    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57"} err="failed to get container status \"82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57\": rpc error: code = NotFound desc = could not find container \"82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57\": container with ID starting with 82686e497a5ae5f80cb5c76bd9b54924725150fa0281beb30ba58ae539f64c57 not found: ID does not exist"
Jul 19 14:55:12.395205 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.395178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fb96885cf-qmjp2"]
Jul 19 14:55:12.397061 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:12.397039    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fb96885cf-qmjp2"]
Jul 19 14:55:13.537419 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:13.537386    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="525501f0-f14f-4212-9f50-f47957ba0e35" path="/var/lib/kubelet/pods/525501f0-f14f-4212-9f50-f47957ba0e35/volumes"
Jul 19 14:55:17.232774 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:17.232743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:17.333748 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:17.333719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:34.571349 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:34.571298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:38.608345 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:38.608307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:39.978401 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:39.978358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:42.200940 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:42.200900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:42.302064 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:42.302021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:43.780710 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:43.780677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:43.877835 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:43.877802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:44.179302 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:44.179224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:44.296066 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:44.296032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:45.790078 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:45.790031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:46.183384 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:46.183311    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:49.192180 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:49.192147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:49.243078 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:49.243040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:51.290064 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:51.290032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:51.393255 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:51.393186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:54.587389 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:54.587350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:55.054296 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.054258    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k2zhv"]
Jul 19 14:55:55.054596 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.054580    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525501f0-f14f-4212-9f50-f47957ba0e35" containerName="console"
Jul 19 14:55:55.054680 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.054600    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="525501f0-f14f-4212-9f50-f47957ba0e35" containerName="console"
Jul 19 14:55:55.054680 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.054665    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="525501f0-f14f-4212-9f50-f47957ba0e35" containerName="console"
Jul 19 14:55:55.057338 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.057317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.059752 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.059731    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 19 14:55:55.060713 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.060694    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 19 14:55:55.060824 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.060708    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jx57k\""
Jul 19 14:55:55.070878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.070853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k2zhv"]
Jul 19 14:55:55.192101 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.192058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgx2d\" (UniqueName: \"kubernetes.io/projected/4fcbda42-4845-4b2b-8114-6faaeb1a2a49-kube-api-access-tgx2d\") pod \"cert-manager-55b4d954d5-k2zhv\" (UID: \"4fcbda42-4845-4b2b-8114-6faaeb1a2a49\") " pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.192101 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.192107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fcbda42-4845-4b2b-8114-6faaeb1a2a49-bound-sa-token\") pod \"cert-manager-55b4d954d5-k2zhv\" (UID: \"4fcbda42-4845-4b2b-8114-6faaeb1a2a49\") " pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.292858 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.292820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fcbda42-4845-4b2b-8114-6faaeb1a2a49-bound-sa-token\") pod \"cert-manager-55b4d954d5-k2zhv\" (UID: \"4fcbda42-4845-4b2b-8114-6faaeb1a2a49\") " pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.292979 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.292898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgx2d\" (UniqueName: \"kubernetes.io/projected/4fcbda42-4845-4b2b-8114-6faaeb1a2a49-kube-api-access-tgx2d\") pod \"cert-manager-55b4d954d5-k2zhv\" (UID: \"4fcbda42-4845-4b2b-8114-6faaeb1a2a49\") " pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.300840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.300806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fcbda42-4845-4b2b-8114-6faaeb1a2a49-bound-sa-token\") pod \"cert-manager-55b4d954d5-k2zhv\" (UID: \"4fcbda42-4845-4b2b-8114-6faaeb1a2a49\") " pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.300958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.300866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgx2d\" (UniqueName: \"kubernetes.io/projected/4fcbda42-4845-4b2b-8114-6faaeb1a2a49-kube-api-access-tgx2d\") pod \"cert-manager-55b4d954d5-k2zhv\" (UID: \"4fcbda42-4845-4b2b-8114-6faaeb1a2a49\") " pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.365862 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.365776    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k2zhv"
Jul 19 14:55:55.494313 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.494285    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k2zhv"]
Jul 19 14:55:55.496773 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:55:55.496737    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fcbda42_4845_4b2b_8114_6faaeb1a2a49.slice/crio-6e505a48a61c9243d9c79e9c9caaff99e49277f7ae9af2f6ba529ca88314d893 WatchSource:0}: Error finding container 6e505a48a61c9243d9c79e9c9caaff99e49277f7ae9af2f6ba529ca88314d893: Status 404 returned error can't find the container with id 6e505a48a61c9243d9c79e9c9caaff99e49277f7ae9af2f6ba529ca88314d893
Jul 19 14:55:55.524027 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.523995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k2zhv" event={"ID":"4fcbda42-4845-4b2b-8114-6faaeb1a2a49","Type":"ContainerStarted","Data":"6e505a48a61c9243d9c79e9c9caaff99e49277f7ae9af2f6ba529ca88314d893"}
Jul 19 14:55:55.644515 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.644412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:55.742569 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:55.742538    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:57.636232 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.636199    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"]
Jul 19 14:55:57.639506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.639471    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.642125 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.642076    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:55:57.642255 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.642126    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 19 14:55:57.643022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.643000    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zsv72\""
Jul 19 14:55:57.650422 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.650392    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"]
Jul 19 14:55:57.816110 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.816067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbfm\" (UniqueName: \"kubernetes.io/projected/3717f16a-c5b0-46bf-9468-f7f7128fe3ec-kube-api-access-ghbfm\") pod \"openshift-lws-operator-fd8ccff4c-sfvmm\" (UID: \"3717f16a-c5b0-46bf-9468-f7f7128fe3ec\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.816288 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.816127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3717f16a-c5b0-46bf-9468-f7f7128fe3ec-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sfvmm\" (UID: \"3717f16a-c5b0-46bf-9468-f7f7128fe3ec\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.917544 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.917441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3717f16a-c5b0-46bf-9468-f7f7128fe3ec-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sfvmm\" (UID: \"3717f16a-c5b0-46bf-9468-f7f7128fe3ec\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.917544 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.917533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbfm\" (UniqueName: \"kubernetes.io/projected/3717f16a-c5b0-46bf-9468-f7f7128fe3ec-kube-api-access-ghbfm\") pod \"openshift-lws-operator-fd8ccff4c-sfvmm\" (UID: \"3717f16a-c5b0-46bf-9468-f7f7128fe3ec\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.917909 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.917878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3717f16a-c5b0-46bf-9468-f7f7128fe3ec-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sfvmm\" (UID: \"3717f16a-c5b0-46bf-9468-f7f7128fe3ec\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.926080 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.926050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbfm\" (UniqueName: \"kubernetes.io/projected/3717f16a-c5b0-46bf-9468-f7f7128fe3ec-kube-api-access-ghbfm\") pod \"openshift-lws-operator-fd8ccff4c-sfvmm\" (UID: \"3717f16a-c5b0-46bf-9468-f7f7128fe3ec\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:57.949692 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:57.949658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"
Jul 19 14:55:58.438116 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:58.438091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm"]
Jul 19 14:55:58.440619 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:55:58.440595    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3717f16a_c5b0_46bf_9468_f7f7128fe3ec.slice/crio-efe2145bf0b3976bb634e20df1ee20cd16ebd7ab38ce50d6dc90e0cfbf6f19fa WatchSource:0}: Error finding container efe2145bf0b3976bb634e20df1ee20cd16ebd7ab38ce50d6dc90e0cfbf6f19fa: Status 404 returned error can't find the container with id efe2145bf0b3976bb634e20df1ee20cd16ebd7ab38ce50d6dc90e0cfbf6f19fa
Jul 19 14:55:58.536788 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:58.536755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k2zhv" event={"ID":"4fcbda42-4845-4b2b-8114-6faaeb1a2a49","Type":"ContainerStarted","Data":"861d766a956703f94dbe8d406b79afe1acfa0c079f6dc0aaf4a92a169a9fab39"}
Jul 19 14:55:58.537999 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:58.537978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm" event={"ID":"3717f16a-c5b0-46bf-9468-f7f7128fe3ec","Type":"ContainerStarted","Data":"efe2145bf0b3976bb634e20df1ee20cd16ebd7ab38ce50d6dc90e0cfbf6f19fa"}
Jul 19 14:55:58.552484 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:55:58.552444    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-k2zhv" podStartSLOduration=0.667005758 podStartE2EDuration="3.552429406s" totalImagesPullingTime="2.885423648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:55:58.552070491 +0000 UTC m=+539.577079565" watchObservedRunningTime="2026-07-19 14:55:58.552429406 +0000 UTC m=+539.577438481"
Jul 19 14:56:01.549814 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:01.549777    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm" event={"ID":"3717f16a-c5b0-46bf-9468-f7f7128fe3ec","Type":"ContainerStarted","Data":"65956407a9cdd641ae6071c3380a13cd6c677f21b288808d4c42fc2938a8bab9"}
Jul 19 14:56:01.564703 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:01.564661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sfvmm" podStartSLOduration=2.354361378 podStartE2EDuration="4.56464664s" totalImagesPullingTime="2.210285262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:01.563831387 +0000 UTC m=+542.588840487" watchObservedRunningTime="2026-07-19 14:56:01.56464664 +0000 UTC m=+542.589655714"
Jul 19 14:56:02.295456 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:02.295414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:02.395944 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:02.395910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:04.406429 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:04.406395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:17.012456 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.012417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:17.105610 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.105574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:17.618224 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.618192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:17.824012 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.823982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:17.924812 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.924738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:17.972464 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.972437    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"]
Jul 19 14:56:17.979096 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.979080    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:17.981728 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.981702    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 19 14:56:17.981868 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.981757    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 19 14:56:17.981868 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.981702    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 19 14:56:17.982042 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.982026    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 19 14:56:17.982115 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.982059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2qtz6\""
Jul 19 14:56:17.990477 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:17.990456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"]
Jul 19 14:56:18.058618 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.058590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5fxp\" (UniqueName: \"kubernetes.io/projected/fb14947d-0be7-4f6a-9ee6-df15c29672ca-kube-api-access-x5fxp\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.059031 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.058634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb14947d-0be7-4f6a-9ee6-df15c29672ca-tmp\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.059031 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.058740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb14947d-0be7-4f6a-9ee6-df15c29672ca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.059031 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.058792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb14947d-0be7-4f6a-9ee6-df15c29672ca-manifests\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.059031 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.058829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb14947d-0be7-4f6a-9ee6-df15c29672ca-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.159489 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.159454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb14947d-0be7-4f6a-9ee6-df15c29672ca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.159675 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.159528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb14947d-0be7-4f6a-9ee6-df15c29672ca-manifests\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.159675 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.159551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb14947d-0be7-4f6a-9ee6-df15c29672ca-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.159675 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.159584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5fxp\" (UniqueName: \"kubernetes.io/projected/fb14947d-0be7-4f6a-9ee6-df15c29672ca-kube-api-access-x5fxp\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.159675 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.159618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb14947d-0be7-4f6a-9ee6-df15c29672ca-tmp\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.159988 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.159965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb14947d-0be7-4f6a-9ee6-df15c29672ca-manifests\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.161778 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.161747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb14947d-0be7-4f6a-9ee6-df15c29672ca-tmp\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.161948 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.161929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb14947d-0be7-4f6a-9ee6-df15c29672ca-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.162009 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.161955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb14947d-0be7-4f6a-9ee6-df15c29672ca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.165738 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.165715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.169815 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.169795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5fxp\" (UniqueName: \"kubernetes.io/projected/fb14947d-0be7-4f6a-9ee6-df15c29672ca-kube-api-access-x5fxp\") pod \"opendatahub-operator-controller-manager-c7885d785-6n4sq\" (UID: \"fb14947d-0be7-4f6a-9ee6-df15c29672ca\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.263696 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.263659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.288629 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.288604    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:18.364126 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.364098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.420793 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.420767    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"]
Jul 19 14:56:18.423011 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:56:18.422985    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb14947d_0be7_4f6a_9ee6_df15c29672ca.slice/crio-7603d2e62463f90c76efe7208677b8f0a3e31b6ccbce671878445927e8373e96 WatchSource:0}: Error finding container 7603d2e62463f90c76efe7208677b8f0a3e31b6ccbce671878445927e8373e96: Status 404 returned error can't find the container with id 7603d2e62463f90c76efe7208677b8f0a3e31b6ccbce671878445927e8373e96
Jul 19 14:56:18.464167 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.464130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.565589 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.565509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.609392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.609371    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq" event={"ID":"fb14947d-0be7-4f6a-9ee6-df15c29672ca","Type":"ContainerStarted","Data":"7603d2e62463f90c76efe7208677b8f0a3e31b6ccbce671878445927e8373e96"}
Jul 19 14:56:18.665467 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.665446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.764278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.764255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:18.882928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:18.882848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:21.624022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:21.623926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq" event={"ID":"fb14947d-0be7-4f6a-9ee6-df15c29672ca","Type":"ContainerStarted","Data":"22696332f51da5e376bacf6bb14f9e8e27dbc0ea0483e54dff4152440526e931"}
Jul 19 14:56:22.629598 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:22.629566    2577 generic.go:356] "Generic (PLEG): container finished" podID="fb14947d-0be7-4f6a-9ee6-df15c29672ca" containerID="22696332f51da5e376bacf6bb14f9e8e27dbc0ea0483e54dff4152440526e931" exitCode=0
Jul 19 14:56:22.630016 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:22.629656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq" event={"ID":"fb14947d-0be7-4f6a-9ee6-df15c29672ca","Type":"ContainerDied","Data":"22696332f51da5e376bacf6bb14f9e8e27dbc0ea0483e54dff4152440526e931"}
Jul 19 14:56:23.636252 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:23.636215    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq" event={"ID":"fb14947d-0be7-4f6a-9ee6-df15c29672ca","Type":"ContainerStarted","Data":"0d8e12f656e4d5d1618fe33703fc10301c4887ca8c744d0406c0059178942843"}
Jul 19 14:56:23.636678 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:23.636349    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:23.661029 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:23.660977    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq" podStartSLOduration=2.32333861 podStartE2EDuration="6.660956067s" totalImagesPullingTime="3.320918292s" totalInitContainerRuntime="1.016699165s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:23.658188522 +0000 UTC m=+564.683197608" watchObservedRunningTime="2026-07-19 14:56:23.660956067 +0000 UTC m=+564.685965143"
Jul 19 14:56:29.152116 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:29.152086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:30.299567 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:30.299533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:31.690727 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:31.690696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:31.792134 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:31.792100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:32.921657 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:32.921627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:34.642620 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:34.642593    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-6n4sq"
Jul 19 14:56:36.650295 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:36.650266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:36.798591 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:36.798556    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:37.342724 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:37.342686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:37.447874 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:37.447836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:37.827762 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:37.827720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:37.910363 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:37.910328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:38.011343 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:38.011309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:39.782156 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.782116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:39.905686 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.905653    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"]
Jul 19 14:56:39.913766 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.913743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:39.916694 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.916673    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 19 14:56:39.916694 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.916691    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 19 14:56:39.917829 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.917807    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 19 14:56:39.918398 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.918007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w47qz\""
Jul 19 14:56:39.918519 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.918050    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 19 14:56:39.920301 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:39.920278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"]
Jul 19 14:56:40.019400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.019362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4e16d34-9332-4118-9f73-5a896e0cdda5-tmp\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.019400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.019401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnqs8\" (UniqueName: \"kubernetes.io/projected/a4e16d34-9332-4118-9f73-5a896e0cdda5-kube-api-access-pnqs8\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.019677 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.019439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4e16d34-9332-4118-9f73-5a896e0cdda5-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.120744 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.120662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4e16d34-9332-4118-9f73-5a896e0cdda5-tmp\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.120744 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.120698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnqs8\" (UniqueName: \"kubernetes.io/projected/a4e16d34-9332-4118-9f73-5a896e0cdda5-kube-api-access-pnqs8\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.120744 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.120721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4e16d34-9332-4118-9f73-5a896e0cdda5-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.120950 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:40.120869    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 19 14:56:40.120950 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:40.120938    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e16d34-9332-4118-9f73-5a896e0cdda5-tls-certs podName:a4e16d34-9332-4118-9f73-5a896e0cdda5 nodeName:}" failed. No retries permitted until 2026-07-19 14:56:40.620919247 +0000 UTC m=+581.645928303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a4e16d34-9332-4118-9f73-5a896e0cdda5-tls-certs") pod "kube-auth-proxy-5ccf86bb4b-4kpsb" (UID: "a4e16d34-9332-4118-9f73-5a896e0cdda5") : secret "kube-auth-proxy-tls" not found
Jul 19 14:56:40.123046 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.123022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4e16d34-9332-4118-9f73-5a896e0cdda5-tmp\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.130056 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.130033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnqs8\" (UniqueName: \"kubernetes.io/projected/a4e16d34-9332-4118-9f73-5a896e0cdda5-kube-api-access-pnqs8\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.323156 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.323118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:40.625215 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.625183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4e16d34-9332-4118-9f73-5a896e0cdda5-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.627612 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.627587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4e16d34-9332-4118-9f73-5a896e0cdda5-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-4kpsb\" (UID: \"a4e16d34-9332-4118-9f73-5a896e0cdda5\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.826333 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.826301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"
Jul 19 14:56:40.947847 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.947818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:40.954769 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.954740    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"]
Jul 19 14:56:40.958137 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.958114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:40.961922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.961719    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 19 14:56:40.961922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.961738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 19 14:56:40.961922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.961795    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 19 14:56:40.961922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.961819    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-x2nq4\""
Jul 19 14:56:40.961922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.961846    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 19 14:56:40.967513 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.967473    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb"]
Jul 19 14:56:40.971173 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:40.971148    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"]
Jul 19 14:56:40.971905 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:56:40.971878    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e16d34_9332_4118_9f73_5a896e0cdda5.slice/crio-fdb95a4d35266f83315b4e236fe77989145da53c9f16a580b795783db027c9c1 WatchSource:0}: Error finding container fdb95a4d35266f83315b4e236fe77989145da53c9f16a580b795783db027c9c1: Status 404 returned error can't find the container with id fdb95a4d35266f83315b4e236fe77989145da53c9f16a580b795783db027c9c1
Jul 19 14:56:41.028200 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81aba5f-88b6-4621-bd09-ef4446cba069-local-certs\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.028200 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-token\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.028506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.028506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.028506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvb4k\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-kube-api-access-nvb4k\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.028506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-cacerts\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.028506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.028443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.047781 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.047755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:41.129773 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.129928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvb4k\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-kube-api-access-nvb4k\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.129928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-cacerts\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.129928 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.130095 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81aba5f-88b6-4621-bd09-ef4446cba069-local-certs\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.130095 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-token\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.130095 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.129984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.130509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.130461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.132849 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.132784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.132958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.132856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81aba5f-88b6-4621-bd09-ef4446cba069-local-certs\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.132958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.132927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.132958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.132941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-cacerts\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.143195 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.143166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-token\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.144229 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.144192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvb4k\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-kube-api-access-nvb4k\") pod \"istiod-openshift-gateway-7574459656-mldtn\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.271016 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.270982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:41.403739 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.403716    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"]
Jul 19 14:56:41.405884 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:56:41.405858    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81aba5f_88b6_4621_bd09_ef4446cba069.slice/crio-83cc3df02ca6c6296f432bb4721e647260e94465d2b85739ee877251984fa9ea WatchSource:0}: Error finding container 83cc3df02ca6c6296f432bb4721e647260e94465d2b85739ee877251984fa9ea: Status 404 returned error can't find the container with id 83cc3df02ca6c6296f432bb4721e647260e94465d2b85739ee877251984fa9ea
Jul 19 14:56:41.706602 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.706516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" event={"ID":"d81aba5f-88b6-4621-bd09-ef4446cba069","Type":"ContainerStarted","Data":"83cc3df02ca6c6296f432bb4721e647260e94465d2b85739ee877251984fa9ea"}
Jul 19 14:56:41.707886 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.707856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb" event={"ID":"a4e16d34-9332-4118-9f73-5a896e0cdda5","Type":"ContainerStarted","Data":"fdb95a4d35266f83315b4e236fe77989145da53c9f16a580b795783db027c9c1"}
Jul 19 14:56:41.986294 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:41.986254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:43.721893 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:43.721858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:43.822323 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:43.822290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:44.480825 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.480793    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:56:44.480939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.480853    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:56:44.727278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.727190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb" event={"ID":"a4e16d34-9332-4118-9f73-5a896e0cdda5","Type":"ContainerStarted","Data":"51d880e09b342e1c9460bafa528435d149e984cb957221bc9318d738de2f8dc0"}
Jul 19 14:56:44.728786 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.728760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" event={"ID":"d81aba5f-88b6-4621-bd09-ef4446cba069","Type":"ContainerStarted","Data":"ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b"}
Jul 19 14:56:44.728922 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.728891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:44.758231 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.758180    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-4kpsb" podStartSLOduration=2.254361834 podStartE2EDuration="5.758162484s" totalImagesPullingTime="3.50380065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:44.756285212 +0000 UTC m=+585.781294287" watchObservedRunningTime="2026-07-19 14:56:44.758162484 +0000 UTC m=+585.783171566"
Jul 19 14:56:44.794210 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:44.794157    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" podStartSLOduration=1.721493126 podStartE2EDuration="4.794138515s" totalImagesPullingTime="3.072645389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:44.791975551 +0000 UTC m=+585.816984656" watchObservedRunningTime="2026-07-19 14:56:44.794138515 +0000 UTC m=+585.819147592"
Jul 19 14:56:45.531855 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:45.531800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:45.691315 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:45.691279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:45.734672 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:45.734642    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-mldtn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 14:56:45.735104 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:45.734700    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" podUID="d81aba5f-88b6-4621-bd09-ef4446cba069" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 14:56:46.621451 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.621411    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"]
Jul 19 14:56:46.624858 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.624834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.627742 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.627586    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-cllrm\""
Jul 19 14:56:46.638510 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.638468    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"]
Jul 19 14:56:46.669446 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.669419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:46.681477 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681693 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf056216-dc88-48c4-a21e-8e959362da24-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681693 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf056216-dc88-48c4-a21e-8e959362da24-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681693 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681693 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dktr\" (UniqueName: \"kubernetes.io/projected/cf056216-dc88-48c4-a21e-8e959362da24-kube-api-access-6dktr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681693 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681970 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf056216-dc88-48c4-a21e-8e959362da24-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681970 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.681970 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.681802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783122 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783122 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dktr\" (UniqueName: \"kubernetes.io/projected/cf056216-dc88-48c4-a21e-8e959362da24-kube-api-access-6dktr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf056216-dc88-48c4-a21e-8e959362da24-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf056216-dc88-48c4-a21e-8e959362da24-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf056216-dc88-48c4-a21e-8e959362da24-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783983 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783983 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.783983 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.783843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.784382 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.784361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf056216-dc88-48c4-a21e-8e959362da24-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.785691 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.785661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf056216-dc88-48c4-a21e-8e959362da24-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.786033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.786016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf056216-dc88-48c4-a21e-8e959362da24-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.796349 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.796316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dktr\" (UniqueName: \"kubernetes.io/projected/cf056216-dc88-48c4-a21e-8e959362da24-kube-api-access-6dktr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.796638 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.796622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf056216-dc88-48c4-a21e-8e959362da24-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lpbm5\" (UID: \"cf056216-dc88-48c4-a21e-8e959362da24\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:46.932056 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.931968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:46.935766 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:46.935741    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:47.031368 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.031334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:47.104485 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.104451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"]
Jul 19 14:56:47.107680 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:56:47.107652    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf056216_dc88_48c4_a21e_8e959362da24.slice/crio-0a66800bc19ed47593de31324ae15f44f57b47ac0472de202a7044dc824c1db5 WatchSource:0}: Error finding container 0a66800bc19ed47593de31324ae15f44f57b47ac0472de202a7044dc824c1db5: Status 404 returned error can't find the container with id 0a66800bc19ed47593de31324ae15f44f57b47ac0472de202a7044dc824c1db5
Jul 19 14:56:47.747355 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.747316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5" event={"ID":"cf056216-dc88-48c4-a21e-8e959362da24","Type":"ContainerStarted","Data":"0a66800bc19ed47593de31324ae15f44f57b47ac0472de202a7044dc824c1db5"}
Jul 19 14:56:47.920889 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.920839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:47.988085 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.988034    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"]
Jul 19 14:56:47.991834 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.991800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:47.995486 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.995446    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 19 14:56:47.997920 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:47.997812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6t9d7\""
Jul 19 14:56:48.005270 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.004383    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"]
Jul 19 14:56:48.008612 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.008577    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"]
Jul 19 14:56:48.008832 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.008809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.013924 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.013763    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 19 14:56:48.014269 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.014109    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dm7qq\""
Jul 19 14:56:48.025987 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.025912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:48.027207 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.027183    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"]
Jul 19 14:56:48.093189 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.093138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-786kn\" (UniqueName: \"kubernetes.io/projected/b320f416-e8bb-45d2-a997-ca035bdcb944-kube-api-access-786kn\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.093372 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.093207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvd22\" (UniqueName: \"kubernetes.io/projected/dad499b2-7418-45cf-b851-5ceef13e66d8-kube-api-access-zvd22\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.093372 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.093309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b320f416-e8bb-45d2-a997-ca035bdcb944-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.093372 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.093364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dad499b2-7418-45cf-b851-5ceef13e66d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.193885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvd22\" (UniqueName: \"kubernetes.io/projected/dad499b2-7418-45cf-b851-5ceef13e66d8-kube-api-access-zvd22\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.193953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b320f416-e8bb-45d2-a997-ca035bdcb944-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.193991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dad499b2-7418-45cf-b851-5ceef13e66d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.194071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-786kn\" (UniqueName: \"kubernetes.io/projected/b320f416-e8bb-45d2-a997-ca035bdcb944-kube-api-access-786kn\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:48.194327    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:48.194400    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b320f416-e8bb-45d2-a997-ca035bdcb944-cert podName:b320f416-e8bb-45d2-a997-ca035bdcb944 nodeName:}" failed. No retries permitted until 2026-07-19 14:56:48.694379911 +0000 UTC m=+589.719388984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b320f416-e8bb-45d2-a997-ca035bdcb944-cert") pod "llmisvc-controller-manager-777c6bd7d5-wjrft" (UID: "b320f416-e8bb-45d2-a997-ca035bdcb944") : secret "llmisvc-webhook-server-cert" not found
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:48.194423    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 19 14:56:48.198516 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:48.194474    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dad499b2-7418-45cf-b851-5ceef13e66d8-cert podName:dad499b2-7418-45cf-b851-5ceef13e66d8 nodeName:}" failed. No retries permitted until 2026-07-19 14:56:48.694458103 +0000 UTC m=+589.719467165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dad499b2-7418-45cf-b851-5ceef13e66d8-cert") pod "kserve-controller-manager-d6f4b65b8-54k9g" (UID: "dad499b2-7418-45cf-b851-5ceef13e66d8") : secret "kserve-webhook-server-cert" not found
Jul 19 14:56:48.209749 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.209717    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvd22\" (UniqueName: \"kubernetes.io/projected/dad499b2-7418-45cf-b851-5ceef13e66d8-kube-api-access-zvd22\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.219115 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.219090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-786kn\" (UniqueName: \"kubernetes.io/projected/b320f416-e8bb-45d2-a997-ca035bdcb944-kube-api-access-786kn\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.392528 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.392417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:48.456853 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.456813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:48.700769 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.700688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b320f416-e8bb-45d2-a997-ca035bdcb944-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.700769 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.700731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dad499b2-7418-45cf-b851-5ceef13e66d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.703565 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.703537    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dad499b2-7418-45cf-b851-5ceef13e66d8-cert\") pod \"kserve-controller-manager-d6f4b65b8-54k9g\" (UID: \"dad499b2-7418-45cf-b851-5ceef13e66d8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.703565 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.703553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b320f416-e8bb-45d2-a997-ca035bdcb944-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wjrft\" (UID: \"b320f416-e8bb-45d2-a997-ca035bdcb944\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:48.734017 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.733982    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:48.906783 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.906750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:48.922563 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:48.922534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:49.194205 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.194174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:49.293099 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.293068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:49.394392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.394363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:49.785082 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.785048    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:56:49.785170 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.785137    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:56:49.785208 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.785184    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:56:49.850693 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.850661    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"]
Jul 19 14:56:49.853041 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:56:49.853003    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb320f416_e8bb_45d2_a997_ca035bdcb944.slice/crio-3c2bec232854dc75e53295f17905fc0e8afad469be0b9770c28ad97cfdd5b700 WatchSource:0}: Error finding container 3c2bec232854dc75e53295f17905fc0e8afad469be0b9770c28ad97cfdd5b700: Status 404 returned error can't find the container with id 3c2bec232854dc75e53295f17905fc0e8afad469be0b9770c28ad97cfdd5b700
Jul 19 14:56:49.874784 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:49.874709    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"]
Jul 19 14:56:50.146004 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.145972    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"]
Jul 19 14:56:50.146388 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.146139    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" podUID="d81aba5f-88b6-4621-bd09-ef4446cba069" containerName="discovery" containerID="cri-o://ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b" gracePeriod=30
Jul 19 14:56:50.376192 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.376168    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:50.516489 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81aba5f-88b6-4621-bd09-ef4446cba069-local-certs\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.516489 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvb4k\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-kube-api-access-nvb4k\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.516777 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516563    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-dns-cert\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.516777 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516632    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-token\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.516777 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516659    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-ca-configmap\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.516777 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-cacerts\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.516777 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.516730    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-kubeconfig\") pod \"d81aba5f-88b6-4621-bd09-ef4446cba069\" (UID: \"d81aba5f-88b6-4621-bd09-ef4446cba069\") "
Jul 19 14:56:50.517117 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.517091    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:56:50.518716 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.518689    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d81aba5f-88b6-4621-bd09-ef4446cba069-local-certs" (OuterVolumeSpecName: "local-certs") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:56:50.518818 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.518759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-kube-api-access-nvb4k" (OuterVolumeSpecName: "kube-api-access-nvb4k") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "kube-api-access-nvb4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:56:50.519068 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.519049    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:56:50.519186 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.519169    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-cacerts" (OuterVolumeSpecName: "cacerts") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:56:50.519223 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.519190    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:56:50.519223 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.519190    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-token" (OuterVolumeSpecName: "istio-token") pod "d81aba5f-88b6-4621-bd09-ef4446cba069" (UID: "d81aba5f-88b6-4621-bd09-ef4446cba069"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:56:50.617278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617247    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-dns-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.617278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617271    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.617278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617280    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-csr-ca-configmap\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.617278 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617289    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-cacerts\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.617653 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617297    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d81aba5f-88b6-4621-bd09-ef4446cba069-istio-kubeconfig\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.617653 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617307    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81aba5f-88b6-4621-bd09-ef4446cba069-local-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.617653 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.617315    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvb4k\" (UniqueName: \"kubernetes.io/projected/d81aba5f-88b6-4621-bd09-ef4446cba069-kube-api-access-nvb4k\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:56:50.771121 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.771043    2577 generic.go:356] "Generic (PLEG): container finished" podID="d81aba5f-88b6-4621-bd09-ef4446cba069" containerID="ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b" exitCode=0
Jul 19 14:56:50.771277 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.771192    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"
Jul 19 14:56:50.771277 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.771213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" event={"ID":"d81aba5f-88b6-4621-bd09-ef4446cba069","Type":"ContainerDied","Data":"ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b"}
Jul 19 14:56:50.771392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.771303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mldtn" event={"ID":"d81aba5f-88b6-4621-bd09-ef4446cba069","Type":"ContainerDied","Data":"83cc3df02ca6c6296f432bb4721e647260e94465d2b85739ee877251984fa9ea"}
Jul 19 14:56:50.771392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.771335    2577 scope.go:117] "RemoveContainer" containerID="ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b"
Jul 19 14:56:50.773021 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.772705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g" event={"ID":"dad499b2-7418-45cf-b851-5ceef13e66d8","Type":"ContainerStarted","Data":"4c0c3e001ae2497b9c7b42e9b76828a2d9724a284ea533bdf1c6743f328c33ad"}
Jul 19 14:56:50.775044 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.775013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5" event={"ID":"cf056216-dc88-48c4-a21e-8e959362da24","Type":"ContainerStarted","Data":"5a7481bf2cf1394e413744597ece45ee4abed4ce57b4ee038a9d40286b060926"}
Jul 19 14:56:50.776590 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.776551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft" event={"ID":"b320f416-e8bb-45d2-a997-ca035bdcb944","Type":"ContainerStarted","Data":"3c2bec232854dc75e53295f17905fc0e8afad469be0b9770c28ad97cfdd5b700"}
Jul 19 14:56:50.814588 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.814463    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5" podStartSLOduration=2.139329102 podStartE2EDuration="4.814439952s" totalImagesPullingTime="2.67511085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:50.812880221 +0000 UTC m=+591.837889297" watchObservedRunningTime="2026-07-19 14:56:50.814439952 +0000 UTC m=+591.839449028"
Jul 19 14:56:50.837549 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.837508    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"]
Jul 19 14:56:50.849808 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.849778    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mldtn"]
Jul 19 14:56:50.860881 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.860858    2577 scope.go:117] "RemoveContainer" containerID="ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b"
Jul 19 14:56:50.861187 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:56:50.861166    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b\": container with ID starting with ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b not found: ID does not exist" containerID="ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b"
Jul 19 14:56:50.861255 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.861195    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b"} err="failed to get container status \"ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b\": rpc error: code = NotFound desc = could not find container \"ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b\": container with ID starting with ddd1c64cb49fd044dcb6e3a0719de54c66af51c76c1d6883c7a0ca51aa5a983b not found: ID does not exist"
Jul 19 14:56:50.936297 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.936262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:50.941145 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:50.941118    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:51.541525 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:51.541470    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d81aba5f-88b6-4621-bd09-ef4446cba069" path="/var/lib/kubelet/pods/d81aba5f-88b6-4621-bd09-ef4446cba069/volumes"
Jul 19 14:56:51.782531 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:51.782487    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:51.783762 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:51.783737    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lpbm5"
Jul 19 14:56:53.792981 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:53.792949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft" event={"ID":"b320f416-e8bb-45d2-a997-ca035bdcb944","Type":"ContainerStarted","Data":"53fc35c78f5dd207f338521bc5df4a07cf2ecd7812102d16ad5848dc9ca7aac5"}
Jul 19 14:56:53.793435 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:53.793025    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:56:53.794535 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:53.794490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g" event={"ID":"dad499b2-7418-45cf-b851-5ceef13e66d8","Type":"ContainerStarted","Data":"8112ec7e7c316c6903e6d4035c3d4cdabdf37572a9e43842354dd95015521b94"}
Jul 19 14:56:53.794708 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:53.794690    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:56:53.809570 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:53.809531    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft" podStartSLOduration=3.8910491499999997 podStartE2EDuration="6.809518446s" totalImagesPullingTime="2.918469296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:53.809019743 +0000 UTC m=+594.834028818" watchObservedRunningTime="2026-07-19 14:56:53.809518446 +0000 UTC m=+594.834527518"
Jul 19 14:56:53.825540 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:56:53.825481    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g" podStartSLOduration=3.936927728 podStartE2EDuration="6.825469358s" totalImagesPullingTime="2.88854163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:53.824302475 +0000 UTC m=+594.849311552" watchObservedRunningTime="2026-07-19 14:56:53.825469358 +0000 UTC m=+594.850478433"
Jul 19 14:57:15.002591 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:15.002560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:57:15.406874 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:15.406787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:57:24.802168 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:24.802091    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wjrft"
Jul 19 14:57:24.804959 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:24.804937    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-54k9g"
Jul 19 14:57:38.279328 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.279285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:57:38.374185 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.374111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:57:38.405370 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.405338    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5bdhs"]
Jul 19 14:57:38.405629 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.405617    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d81aba5f-88b6-4621-bd09-ef4446cba069" containerName="discovery"
Jul 19 14:57:38.405683 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.405631    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d81aba5f-88b6-4621-bd09-ef4446cba069" containerName="discovery"
Jul 19 14:57:38.405717 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.405692    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d81aba5f-88b6-4621-bd09-ef4446cba069" containerName="discovery"
Jul 19 14:57:38.412182 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.412160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:38.414721 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.414696    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 19 14:57:38.415748 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.415715    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 19 14:57:38.415748 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.415743    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-psf2h\""
Jul 19 14:57:38.418043 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.417960    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5bdhs"]
Jul 19 14:57:38.457719 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.457695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:57:38.487986 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.487956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsp6c\" (UniqueName: \"kubernetes.io/projected/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca-kube-api-access-fsp6c\") pod \"kuadrant-operator-catalog-5bdhs\" (UID: \"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca\") " pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:38.589396 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.589366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsp6c\" (UniqueName: \"kubernetes.io/projected/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca-kube-api-access-fsp6c\") pod \"kuadrant-operator-catalog-5bdhs\" (UID: \"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca\") " pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:38.596949 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.596918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsp6c\" (UniqueName: \"kubernetes.io/projected/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca-kube-api-access-fsp6c\") pod \"kuadrant-operator-catalog-5bdhs\" (UID: \"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca\") " pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:38.724010 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.723980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:38.765529 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.765483    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5bdhs"]
Jul 19 14:57:38.847718 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.847688    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5bdhs"]
Jul 19 14:57:38.850215 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:57:38.850176    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf3b99c_00e1_4349_aa99_f3a8fbb46dca.slice/crio-10dcad098b399047060d85fef6a488210ac659151fc0331cb67af41c4239507b WatchSource:0}: Error finding container 10dcad098b399047060d85fef6a488210ac659151fc0331cb67af41c4239507b: Status 404 returned error can't find the container with id 10dcad098b399047060d85fef6a488210ac659151fc0331cb67af41c4239507b
Jul 19 14:57:38.974540 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.974420    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvhsr"]
Jul 19 14:57:38.977081 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.977058    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:38.983673 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.983634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs" event={"ID":"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca","Type":"ContainerStarted","Data":"10dcad098b399047060d85fef6a488210ac659151fc0331cb67af41c4239507b"}
Jul 19 14:57:38.984994 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:38.984963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvhsr"]
Jul 19 14:57:39.093553 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:39.093514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwl4v\" (UniqueName: \"kubernetes.io/projected/6df328cc-31c0-483d-b4e9-5d30aaa3a356-kube-api-access-gwl4v\") pod \"kuadrant-operator-catalog-vvhsr\" (UID: \"6df328cc-31c0-483d-b4e9-5d30aaa3a356\") " pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:39.193917 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:39.193872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwl4v\" (UniqueName: \"kubernetes.io/projected/6df328cc-31c0-483d-b4e9-5d30aaa3a356-kube-api-access-gwl4v\") pod \"kuadrant-operator-catalog-vvhsr\" (UID: \"6df328cc-31c0-483d-b4e9-5d30aaa3a356\") " pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:39.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:39.202363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwl4v\" (UniqueName: \"kubernetes.io/projected/6df328cc-31c0-483d-b4e9-5d30aaa3a356-kube-api-access-gwl4v\") pod \"kuadrant-operator-catalog-vvhsr\" (UID: \"6df328cc-31c0-483d-b4e9-5d30aaa3a356\") " pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:39.288163 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:39.288071    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:39.420855 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:39.420825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvhsr"]
Jul 19 14:57:39.422846 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:57:39.422817    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df328cc_31c0_483d_b4e9_5d30aaa3a356.slice/crio-34d2e0ed14fdebc563de69b5864d76f82b3b9e5d82ddc366504e02bf66ace7eb WatchSource:0}: Error finding container 34d2e0ed14fdebc563de69b5864d76f82b3b9e5d82ddc366504e02bf66ace7eb: Status 404 returned error can't find the container with id 34d2e0ed14fdebc563de69b5864d76f82b3b9e5d82ddc366504e02bf66ace7eb
Jul 19 14:57:39.990897 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:39.990857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr" event={"ID":"6df328cc-31c0-483d-b4e9-5d30aaa3a356","Type":"ContainerStarted","Data":"34d2e0ed14fdebc563de69b5864d76f82b3b9e5d82ddc366504e02bf66ace7eb"}
Jul 19 14:57:42.001126 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.001084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr" event={"ID":"6df328cc-31c0-483d-b4e9-5d30aaa3a356","Type":"ContainerStarted","Data":"7bdf6b1bde8b933ed5de9b3f16b8e32b5097c96a2403f04a96e82a30aea6230e"}
Jul 19 14:57:42.002328 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.002301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs" event={"ID":"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca","Type":"ContainerStarted","Data":"96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150"}
Jul 19 14:57:42.002446 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.002426    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs" podUID="9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" containerName="registry-server" containerID="cri-o://96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150" gracePeriod=2
Jul 19 14:57:42.015821 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.015784    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr" podStartSLOduration=2.475023632 podStartE2EDuration="4.015771122s" totalImagesPullingTime="1.54074749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:57:42.014654963 +0000 UTC m=+643.039664038" watchObservedRunningTime="2026-07-19 14:57:42.015771122 +0000 UTC m=+643.040780320"
Jul 19 14:57:42.029506 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.029452    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs" podStartSLOduration=1.919992604 podStartE2EDuration="4.02943858s" totalImagesPullingTime="2.109445976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:57:42.028408495 +0000 UTC m=+643.053417572" watchObservedRunningTime="2026-07-19 14:57:42.02943858 +0000 UTC m=+643.054447655"
Jul 19 14:57:42.243256 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.243234    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:42.319086 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.319015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsp6c\" (UniqueName: \"kubernetes.io/projected/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca-kube-api-access-fsp6c\") pod \"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca\" (UID: \"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca\") "
Jul 19 14:57:42.321069 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.321047    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca-kube-api-access-fsp6c" (OuterVolumeSpecName: "kube-api-access-fsp6c") pod "9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" (UID: "9cf3b99c-00e1-4349-aa99-f3a8fbb46dca"). InnerVolumeSpecName "kube-api-access-fsp6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:57:42.420633 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:42.420601    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsp6c\" (UniqueName: \"kubernetes.io/projected/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca-kube-api-access-fsp6c\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:57:43.008077 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.008044    2577 generic.go:356] "Generic (PLEG): container finished" podID="9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" containerID="96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150" exitCode=0
Jul 19 14:57:43.008599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.008089    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs"
Jul 19 14:57:43.008599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.008126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs" event={"ID":"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca","Type":"ContainerDied","Data":"96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150"}
Jul 19 14:57:43.008599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.008178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5bdhs" event={"ID":"9cf3b99c-00e1-4349-aa99-f3a8fbb46dca","Type":"ContainerDied","Data":"10dcad098b399047060d85fef6a488210ac659151fc0331cb67af41c4239507b"}
Jul 19 14:57:43.008599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.008198    2577 scope.go:117] "RemoveContainer" containerID="96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150"
Jul 19 14:57:43.016780 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.016764    2577 scope.go:117] "RemoveContainer" containerID="96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150"
Jul 19 14:57:43.017051 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:57:43.017032    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150\": container with ID starting with 96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150 not found: ID does not exist" containerID="96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150"
Jul 19 14:57:43.017113 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.017056    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150"} err="failed to get container status \"96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150\": rpc error: code = NotFound desc = could not find container \"96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150\": container with ID starting with 96cb0fc6fd5680a4b75e819cc13557ef66c98012fb8437c3ebcb5d57c21f4150 not found: ID does not exist"
Jul 19 14:57:43.028253 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.028232    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5bdhs"]
Jul 19 14:57:43.031751 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.031727    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5bdhs"]
Jul 19 14:57:43.539020 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:43.538984    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" path="/var/lib/kubelet/pods/9cf3b99c-00e1-4349-aa99-f3a8fbb46dca/volumes"
Jul 19 14:57:49.288920 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:49.288877    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:49.289337 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:49.288955    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:49.311132 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:49.311104    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:57:50.060023 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:57:50.059994    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vvhsr"
Jul 19 14:58:01.221949 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:01.221913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:01.622516 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:01.622451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:02.860608 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:02.860570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:02.960189 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:02.960139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:04.892552 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.892516    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"]
Jul 19 14:58:04.892967 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.892825    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" containerName="registry-server"
Jul 19 14:58:04.892967 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.892836    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" containerName="registry-server"
Jul 19 14:58:04.892967 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.892891    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf3b99c-00e1-4349-aa99-f3a8fbb46dca" containerName="registry-server"
Jul 19 14:58:04.898591 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.898569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:04.901155 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.901121    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2dcxw\""
Jul 19 14:58:04.907739 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:04.907707    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"]
Jul 19 14:58:05.016725 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.016691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x24x\" (UniqueName: \"kubernetes.io/projected/f32c0840-2442-446e-a763-7b97b16bd6d9-kube-api-access-7x24x\") pod \"authorino-operator-75cc8d5cb7-m5dr6\" (UID: \"f32c0840-2442-446e-a763-7b97b16bd6d9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:05.117401 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.117362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x24x\" (UniqueName: \"kubernetes.io/projected/f32c0840-2442-446e-a763-7b97b16bd6d9-kube-api-access-7x24x\") pod \"authorino-operator-75cc8d5cb7-m5dr6\" (UID: \"f32c0840-2442-446e-a763-7b97b16bd6d9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:05.130510 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.130459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x24x\" (UniqueName: \"kubernetes.io/projected/f32c0840-2442-446e-a763-7b97b16bd6d9-kube-api-access-7x24x\") pod \"authorino-operator-75cc8d5cb7-m5dr6\" (UID: \"f32c0840-2442-446e-a763-7b97b16bd6d9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:05.210097 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.210005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:05.344678 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.344654    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"]
Jul 19 14:58:05.347265 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:58:05.347235    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32c0840_2442_446e_a763_7b97b16bd6d9.slice/crio-7d4999e80fbcee2774d4ff6bd5a3548cafcfee8ec2243cbe5f41a7e8a1b94446 WatchSource:0}: Error finding container 7d4999e80fbcee2774d4ff6bd5a3548cafcfee8ec2243cbe5f41a7e8a1b94446: Status 404 returned error can't find the container with id 7d4999e80fbcee2774d4ff6bd5a3548cafcfee8ec2243cbe5f41a7e8a1b94446
Jul 19 14:58:05.852940 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.852893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:05.951812 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:05.951735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:06.110088 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:06.109962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6" event={"ID":"f32c0840-2442-446e-a763-7b97b16bd6d9","Type":"ContainerStarted","Data":"7d4999e80fbcee2774d4ff6bd5a3548cafcfee8ec2243cbe5f41a7e8a1b94446"}
Jul 19 14:58:08.119602 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:08.119566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6" event={"ID":"f32c0840-2442-446e-a763-7b97b16bd6d9","Type":"ContainerStarted","Data":"29b052a53bef5a1c611eebda75bdcf4d5ff5a5d1f31a2d45613202ba9bbe27e8"}
Jul 19 14:58:08.119983 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:08.119633    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:08.139627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:08.139576    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6" podStartSLOduration=1.970247251 podStartE2EDuration="4.139561133s" totalImagesPullingTime="2.169313882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:08.137753462 +0000 UTC m=+669.162762550" watchObservedRunningTime="2026-07-19 14:58:08.139561133 +0000 UTC m=+669.164570267"
Jul 19 14:58:09.197106 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.197067    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"]
Jul 19 14:58:09.200333 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.200316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.202979 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.202961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hjf4n\""
Jul 19 14:58:09.213103 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.213080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"]
Jul 19 14:58:09.354708 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.354661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.354907 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.354730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t5rf\" (UniqueName: \"kubernetes.io/projected/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-kube-api-access-9t5rf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.455984 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.455903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.455984 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.455957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t5rf\" (UniqueName: \"kubernetes.io/projected/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-kube-api-access-9t5rf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.456330 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.456308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.466914 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.466890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t5rf\" (UniqueName: \"kubernetes.io/projected/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-kube-api-access-9t5rf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.509964 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.509942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:09.644182 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.644146    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"]
Jul 19 14:58:09.647790 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:58:09.647767    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dfd3b1_724d_4f2d_b52a_f350b60535bb.slice/crio-5b08868bee170320149bcf3aa6f5b55f4242ac2abba3e80e3fed31166d8c33e4 WatchSource:0}: Error finding container 5b08868bee170320149bcf3aa6f5b55f4242ac2abba3e80e3fed31166d8c33e4: Status 404 returned error can't find the container with id 5b08868bee170320149bcf3aa6f5b55f4242ac2abba3e80e3fed31166d8c33e4
Jul 19 14:58:09.842878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:09.842834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:10.128994 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:10.128915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh" event={"ID":"a4dfd3b1-724d-4f2d-b52a-f350b60535bb","Type":"ContainerStarted","Data":"5b08868bee170320149bcf3aa6f5b55f4242ac2abba3e80e3fed31166d8c33e4"}
Jul 19 14:58:11.271921 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.271886    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"]
Jul 19 14:58:11.276017 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.275993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:11.278545 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.278519    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 19 14:58:11.278672 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.278554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hlskh\""
Jul 19 14:58:11.286117 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.286090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"]
Jul 19 14:58:11.371932 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.371898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5zw8\" (UniqueName: \"kubernetes.io/projected/4342cb54-d9a9-44e8-b7d6-f1c897fe93a8-kube-api-access-m5zw8\") pod \"dns-operator-controller-manager-864dd78c4c-pknc5\" (UID: \"4342cb54-d9a9-44e8-b7d6-f1c897fe93a8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:11.473254 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.473212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5zw8\" (UniqueName: \"kubernetes.io/projected/4342cb54-d9a9-44e8-b7d6-f1c897fe93a8-kube-api-access-m5zw8\") pod \"dns-operator-controller-manager-864dd78c4c-pknc5\" (UID: \"4342cb54-d9a9-44e8-b7d6-f1c897fe93a8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:11.481681 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.481655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5zw8\" (UniqueName: \"kubernetes.io/projected/4342cb54-d9a9-44e8-b7d6-f1c897fe93a8-kube-api-access-m5zw8\") pod \"dns-operator-controller-manager-864dd78c4c-pknc5\" (UID: \"4342cb54-d9a9-44e8-b7d6-f1c897fe93a8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:11.587670 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.587595    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:11.744166 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.744139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"]
Jul 19 14:58:11.746822 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:58:11.746793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4342cb54_d9a9_44e8_b7d6_f1c897fe93a8.slice/crio-4162131f7489d57c2fe3afea83a322de98f63384fe0bce45e91974df83368fc6 WatchSource:0}: Error finding container 4162131f7489d57c2fe3afea83a322de98f63384fe0bce45e91974df83368fc6: Status 404 returned error can't find the container with id 4162131f7489d57c2fe3afea83a322de98f63384fe0bce45e91974df83368fc6
Jul 19 14:58:11.848024 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:11.847947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:12.070779 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:12.070741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:12.138594 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:12.138512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5" event={"ID":"4342cb54-d9a9-44e8-b7d6-f1c897fe93a8","Type":"ContainerStarted","Data":"4162131f7489d57c2fe3afea83a322de98f63384fe0bce45e91974df83368fc6"}
Jul 19 14:58:13.467016 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.466972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:13.565875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.565793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:13.780430 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.780393    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"]
Jul 19 14:58:13.784150 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.784128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:13.787336 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.787316    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bk2t6\""
Jul 19 14:58:13.794937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.794913    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"]
Jul 19 14:58:13.897194 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.897157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vkch\" (UniqueName: \"kubernetes.io/projected/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176-kube-api-access-7vkch\") pod \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" (UID: \"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:13.998340 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:13.998307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vkch\" (UniqueName: \"kubernetes.io/projected/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176-kube-api-access-7vkch\") pod \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" (UID: \"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:14.006256 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:14.006229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vkch\" (UniqueName: \"kubernetes.io/projected/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176-kube-api-access-7vkch\") pod \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" (UID: \"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:14.097052 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:14.096965    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:14.465381 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:14.465279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:14.657740 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:14.654323    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"]
Jul 19 14:58:14.659814 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:14.659792    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:58:15.156950 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.156917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5" event={"ID":"4342cb54-d9a9-44e8-b7d6-f1c897fe93a8","Type":"ContainerStarted","Data":"3b79a7f7b67e6aec5aeed780cc90bda412081bd21db957d649ede670e3bcc494"}
Jul 19 14:58:15.157365 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.157015    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:15.158549 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.158521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh" event={"ID":"a4dfd3b1-724d-4f2d-b52a-f350b60535bb","Type":"ContainerStarted","Data":"fa803390f24a123b2da58dfd56eb43aee5c44f8800106024c4dc46eede9bec75"}
Jul 19 14:58:15.158665 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.158585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:15.159725 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.159699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz" event={"ID":"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176","Type":"ContainerStarted","Data":"22eba09ea8d040b95358d37eebe68b7379a29bd0197b13780f32d9f30cac2995"}
Jul 19 14:58:15.195560 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.195519    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5" podStartSLOduration=1.427849246 podStartE2EDuration="4.195483109s" totalImagesPullingTime="2.767633863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:15.177974742 +0000 UTC m=+676.202983898" watchObservedRunningTime="2026-07-19 14:58:15.195483109 +0000 UTC m=+676.220492214"
Jul 19 14:58:15.196388 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.196360    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh" podStartSLOduration=1.312281844 podStartE2EDuration="6.196350522s" totalImagesPullingTime="4.884068678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:15.194971536 +0000 UTC m=+676.219980611" watchObservedRunningTime="2026-07-19 14:58:15.196350522 +0000 UTC m=+676.221359597"
Jul 19 14:58:15.468666 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.468587    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:15.569741 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.569705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:15.670217 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.670183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:15.735589 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:15.735524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:16.026815 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:16.026707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:16.088148 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:16.088112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:17.169312 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:17.169281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz" event={"ID":"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176","Type":"ContainerStarted","Data":"5410a7a40c537fcde5e818995cd6ce9c7900ef86ab53f63a9565a7d61abba995"}
Jul 19 14:58:17.169719 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:17.169423    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:17.187925 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:17.187886    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz" podStartSLOduration=2.642360373 podStartE2EDuration="4.187874177s" totalImagesPullingTime="1.545513804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:17.186658374 +0000 UTC m=+678.211667452" watchObservedRunningTime="2026-07-19 14:58:17.187874177 +0000 UTC m=+678.212883253"
Jul 19 14:58:17.427432 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:17.427341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:17.559619 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:17.559586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:19.126655 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:19.126625    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m5dr6"
Jul 19 14:58:19.658543 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:19.658511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:19.758981 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:19.758935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:24.056858 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:24.056825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:24.158459 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:24.158425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:25.051340 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:25.051302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:25.109095 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:25.109056    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:26.165507 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:26.165461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pknc5"
Jul 19 14:58:26.166071 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:26.165577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:26.664942 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:26.664911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:26.765674 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:26.765640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:28.087682 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.087645    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"]
Jul 19 14:58:28.088102 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.087832    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" containerName="manager" containerID="cri-o://fa803390f24a123b2da58dfd56eb43aee5c44f8800106024c4dc46eede9bec75" gracePeriod=2
Jul 19 14:58:28.093114 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.093080    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"]
Jul 19 14:58:28.104570 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.104347    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"]
Jul 19 14:58:28.104730 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.104652    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" containerName="manager" containerID="cri-o://5410a7a40c537fcde5e818995cd6ce9c7900ef86ab53f63a9565a7d61abba995" gracePeriod=2
Jul 19 14:58:28.107529 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.107464    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.107851 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.107796    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.108782 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.108754    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"]
Jul 19 14:58:28.109782 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.109757    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.112602 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.112577    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"]
Jul 19 14:58:28.112723 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.112690    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.113021 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.113002    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" containerName="manager"
Jul 19 14:58:28.113083 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.113023    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" containerName="manager"
Jul 19 14:58:28.113083 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.113056    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" containerName="manager"
Jul 19 14:58:28.113083 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.113065    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" containerName="manager"
Jul 19 14:58:28.113183 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.113135    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" containerName="manager"
Jul 19 14:58:28.113183 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.113147    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" containerName="manager"
Jul 19 14:58:28.114460 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.114428    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.116200 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.116184    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.129368 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.129339    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"]
Jul 19 14:58:28.132875 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.132853    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"]
Jul 19 14:58:28.136570 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.136538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:28.149650 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.149615    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.150043 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.150021    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"]
Jul 19 14:58:28.151623 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.151593    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.153554 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.153524    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.155433 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.155412    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.201307 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.201281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:28.205428 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.205409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88nn\" (UniqueName: \"kubernetes.io/projected/dab0e4ce-dd73-479f-b438-9d0cd0b3e950-kube-api-access-t88nn\") pod \"limitador-operator-controller-manager-859d7b78bd-bfc9t\" (UID: \"dab0e4ce-dd73-479f-b438-9d0cd0b3e950\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:28.205508 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.205444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fl5fx\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.205508 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.205461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qt8l\" (UniqueName: \"kubernetes.io/projected/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-kube-api-access-8qt8l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fl5fx\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.220796 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.220658    2577 generic.go:356] "Generic (PLEG): container finished" podID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" containerID="5410a7a40c537fcde5e818995cd6ce9c7900ef86ab53f63a9565a7d61abba995" exitCode=0
Jul 19 14:58:28.223378 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.223338    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" containerID="fa803390f24a123b2da58dfd56eb43aee5c44f8800106024c4dc46eede9bec75" exitCode=0
Jul 19 14:58:28.298070 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.298019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:28.306134 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.306108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88nn\" (UniqueName: \"kubernetes.io/projected/dab0e4ce-dd73-479f-b438-9d0cd0b3e950-kube-api-access-t88nn\") pod \"limitador-operator-controller-manager-859d7b78bd-bfc9t\" (UID: \"dab0e4ce-dd73-479f-b438-9d0cd0b3e950\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:28.306268 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.306147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fl5fx\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.306268 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.306166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qt8l\" (UniqueName: \"kubernetes.io/projected/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-kube-api-access-8qt8l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fl5fx\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.306607 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.306585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fl5fx\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.316040 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.316001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qt8l\" (UniqueName: \"kubernetes.io/projected/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-kube-api-access-8qt8l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fl5fx\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.316202 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.316179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88nn\" (UniqueName: \"kubernetes.io/projected/dab0e4ce-dd73-479f-b438-9d0cd0b3e950-kube-api-access-t88nn\") pod \"limitador-operator-controller-manager-859d7b78bd-bfc9t\" (UID: \"dab0e4ce-dd73-479f-b438-9d0cd0b3e950\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:28.351169 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.351099    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.353360 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.353329    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.353612 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.353597    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.355196 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.355176    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.356930 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.356912    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:28.358766 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:28.358744    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:28.407154 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.407126    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vkch\" (UniqueName: \"kubernetes.io/projected/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176-kube-api-access-7vkch\") pod \"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176\" (UID: \"9f0c8ddb-3ab8-4713-85ce-d0ea7a943176\") "
Jul 19 14:58:28.407308 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.407174    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-extensions-socket-volume\") pod \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") "
Jul 19 14:58:28.407308 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.407235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t5rf\" (UniqueName: \"kubernetes.io/projected/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-kube-api-access-9t5rf\") pod \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\" (UID: \"a4dfd3b1-724d-4f2d-b52a-f350b60535bb\") "
Jul 19 14:58:28.407701 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.407676    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a4dfd3b1-724d-4f2d-b52a-f350b60535bb" (UID: "a4dfd3b1-724d-4f2d-b52a-f350b60535bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:58:28.409287 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.409262    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176-kube-api-access-7vkch" (OuterVolumeSpecName: "kube-api-access-7vkch") pod "9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" (UID: "9f0c8ddb-3ab8-4713-85ce-d0ea7a943176"). InnerVolumeSpecName "kube-api-access-7vkch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:58:28.409368 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.409279    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-kube-api-access-9t5rf" (OuterVolumeSpecName: "kube-api-access-9t5rf") pod "a4dfd3b1-724d-4f2d-b52a-f350b60535bb" (UID: "a4dfd3b1-724d-4f2d-b52a-f350b60535bb"). InnerVolumeSpecName "kube-api-access-9t5rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:58:28.507869 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.507833    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t5rf\" (UniqueName: \"kubernetes.io/projected/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-kube-api-access-9t5rf\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:58:28.507869 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.507864    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vkch\" (UniqueName: \"kubernetes.io/projected/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176-kube-api-access-7vkch\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:58:28.508066 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.507878    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dfd3b1-724d-4f2d-b52a-f350b60535bb-extensions-socket-volume\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:58:28.519822 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.519792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:28.527899 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.527875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:28.671878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.671853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"]
Jul 19 14:58:28.674145 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:58:28.674116    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a34cb1_83da_43ed_90b6_aba6fb0d4f14.slice/crio-44b409a160b387acde0ba6719ee2c08ef2d0477c755a496d8d447f9336ec2e3a WatchSource:0}: Error finding container 44b409a160b387acde0ba6719ee2c08ef2d0477c755a496d8d447f9336ec2e3a: Status 404 returned error can't find the container with id 44b409a160b387acde0ba6719ee2c08ef2d0477c755a496d8d447f9336ec2e3a
Jul 19 14:58:28.685720 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:28.685676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"]
Jul 19 14:58:28.687804 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:58:28.687781    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab0e4ce_dd73_479f_b438_9d0cd0b3e950.slice/crio-18ce07c1e6a04edf072697c42841a15eba0b10ca32bbe23eb7e8c6572b56f92e WatchSource:0}: Error finding container 18ce07c1e6a04edf072697c42841a15eba0b10ca32bbe23eb7e8c6572b56f92e: Status 404 returned error can't find the container with id 18ce07c1e6a04edf072697c42841a15eba0b10ca32bbe23eb7e8c6572b56f92e
Jul 19 14:58:29.229558 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.229529    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:29.230008 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.229544    2577 scope.go:117] "RemoveContainer" containerID="5410a7a40c537fcde5e818995cd6ce9c7900ef86ab53f63a9565a7d61abba995"
Jul 19 14:58:29.231153 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.231121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t" event={"ID":"dab0e4ce-dd73-479f-b438-9d0cd0b3e950","Type":"ContainerStarted","Data":"2c3cb32ea3cb82eb6c61f09c8ad319f514b01ec8f6eb1b243d9123016e8eb86b"}
Jul 19 14:58:29.231272 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.231170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t" event={"ID":"dab0e4ce-dd73-479f-b438-9d0cd0b3e950","Type":"ContainerStarted","Data":"18ce07c1e6a04edf072697c42841a15eba0b10ca32bbe23eb7e8c6572b56f92e"}
Jul 19 14:58:29.231272 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.231203    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:29.231787 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.231757    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:29.232826 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.232802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx" event={"ID":"10a34cb1-83da-43ed-90b6-aba6fb0d4f14","Type":"ContainerStarted","Data":"275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e"}
Jul 19 14:58:29.232929 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.232838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx" event={"ID":"10a34cb1-83da-43ed-90b6-aba6fb0d4f14","Type":"ContainerStarted","Data":"44b409a160b387acde0ba6719ee2c08ef2d0477c755a496d8d447f9336ec2e3a"}
Jul 19 14:58:29.232929 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.232888    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:29.233689 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.233661    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:29.234292 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.234276    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:29.238894 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.238745    2577 scope.go:117] "RemoveContainer" containerID="fa803390f24a123b2da58dfd56eb43aee5c44f8800106024c4dc46eede9bec75"
Jul 19 14:58:29.254274 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.254242    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:29.254441 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.254410    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx" podStartSLOduration=1.254397803 podStartE2EDuration="1.254397803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:29.252223621 +0000 UTC m=+690.277232696" watchObservedRunningTime="2026-07-19 14:58:29.254397803 +0000 UTC m=+690.279406879"
Jul 19 14:58:29.256086 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.256064    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:29.276343 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.276298    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t" podStartSLOduration=1.27628538 podStartE2EDuration="1.27628538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:29.275898541 +0000 UTC m=+690.300907616" watchObservedRunningTime="2026-07-19 14:58:29.27628538 +0000 UTC m=+690.301294454"
Jul 19 14:58:29.278043 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.278023    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:29.279927 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.279907    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:29.537643 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.537609    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rpfrh\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rpfrh"
Jul 19 14:58:29.539536 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:29.539478    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kbdfz\" is forbidden: User \"system:node:ip-10-0-142-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-137.ec2.internal' and this object" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbdfz"
Jul 19 14:58:29.539731 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.539716    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f0c8ddb-3ab8-4713-85ce-d0ea7a943176" path="/var/lib/kubelet/pods/9f0c8ddb-3ab8-4713-85ce-d0ea7a943176/volumes"
Jul 19 14:58:29.540077 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:29.540066    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4dfd3b1-724d-4f2d-b52a-f350b60535bb" path="/var/lib/kubelet/pods/a4dfd3b1-724d-4f2d-b52a-f350b60535bb/volumes"
Jul 19 14:58:31.257281 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:31.257245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:31.357694 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:31.357658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.039195 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.039166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.139790 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.139763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.241530 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.241488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.296068 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.295999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.615827 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.615744    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-579bf5847f-bljhq"]
Jul 19 14:58:35.719207 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.719173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.780859 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.780834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:35.847599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:35.847567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:37.718675 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:37.718646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:37.818364 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:37.818334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:40.247627 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:40.247596    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:40.248036 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:40.247775    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bfc9t"
Jul 19 14:58:44.205931 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:44.205900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:44.305958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:44.305933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:45.621367 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:45.621328    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"]
Jul 19 14:58:45.621853 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:45.621550    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx" podUID="10a34cb1-83da-43ed-90b6-aba6fb0d4f14" containerName="manager" containerID="cri-o://275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e" gracePeriod=10
Jul 19 14:58:45.964096 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:45.964066    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:46.067264 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.067232    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-extensions-socket-volume\") pod \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") "
Jul 19 14:58:46.067433 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.067323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qt8l\" (UniqueName: \"kubernetes.io/projected/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-kube-api-access-8qt8l\") pod \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\" (UID: \"10a34cb1-83da-43ed-90b6-aba6fb0d4f14\") "
Jul 19 14:58:46.067642 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.067619    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "10a34cb1-83da-43ed-90b6-aba6fb0d4f14" (UID: "10a34cb1-83da-43ed-90b6-aba6fb0d4f14"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:58:46.069293 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.069274    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-kube-api-access-8qt8l" (OuterVolumeSpecName: "kube-api-access-8qt8l") pod "10a34cb1-83da-43ed-90b6-aba6fb0d4f14" (UID: "10a34cb1-83da-43ed-90b6-aba6fb0d4f14"). InnerVolumeSpecName "kube-api-access-8qt8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:58:46.168163 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.168095    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-extensions-socket-volume\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:58:46.168163 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.168121    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qt8l\" (UniqueName: \"kubernetes.io/projected/10a34cb1-83da-43ed-90b6-aba6fb0d4f14-kube-api-access-8qt8l\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:58:46.322860 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.322825    2577 generic.go:356] "Generic (PLEG): container finished" podID="10a34cb1-83da-43ed-90b6-aba6fb0d4f14" containerID="275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e" exitCode=0
Jul 19 14:58:46.323040 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.322871    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"
Jul 19 14:58:46.323040 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.322900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx" event={"ID":"10a34cb1-83da-43ed-90b6-aba6fb0d4f14","Type":"ContainerDied","Data":"275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e"}
Jul 19 14:58:46.323040 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.322950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx" event={"ID":"10a34cb1-83da-43ed-90b6-aba6fb0d4f14","Type":"ContainerDied","Data":"44b409a160b387acde0ba6719ee2c08ef2d0477c755a496d8d447f9336ec2e3a"}
Jul 19 14:58:46.323040 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.322989    2577 scope.go:117] "RemoveContainer" containerID="275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e"
Jul 19 14:58:46.331607 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.331587    2577 scope.go:117] "RemoveContainer" containerID="275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e"
Jul 19 14:58:46.331871 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:58:46.331836    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e\": container with ID starting with 275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e not found: ID does not exist" containerID="275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e"
Jul 19 14:58:46.331871 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.331865    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e"} err="failed to get container status \"275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e\": rpc error: code = NotFound desc = could not find container \"275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e\": container with ID starting with 275ed1a7c081349ee37dde1b5bf4d0b3c80d68e35714490390f997c7848f523e not found: ID does not exist"
Jul 19 14:58:46.343785 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.343765    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"]
Jul 19 14:58:46.347619 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:46.347600    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fl5fx"]
Jul 19 14:58:47.539101 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:47.539069    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a34cb1-83da-43ed-90b6-aba6fb0d4f14" path="/var/lib/kubelet/pods/10a34cb1-83da-43ed-90b6-aba6fb0d4f14/volumes"
Jul 19 14:58:48.950925 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:48.950893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:49.052903 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:49.052867    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:52.758579 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:52.758481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:52.808810 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:52.808777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:53.358708 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:53.358677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:53.410338 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:53.410309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:56.961518 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:56.961460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:57.061635 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:58:57.061602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:00.635979 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.635933    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-579bf5847f-bljhq" podUID="70308b9b-72bb-4fe1-8833-505cdfa99e1b" containerName="console" containerID="cri-o://197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3" gracePeriod=15
Jul 19 14:59:00.885140 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.885118    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:59:00.992692 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-config\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.992840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg8sd\" (UniqueName: \"kubernetes.io/projected/70308b9b-72bb-4fe1-8833-505cdfa99e1b-kube-api-access-hg8sd\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.992840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-oauth-config\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.992840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992746    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-oauth-serving-cert\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.992840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992770    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-service-ca\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.993030 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-trusted-ca-bundle\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.993030 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.992910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-serving-cert\") pod \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\" (UID: \"70308b9b-72bb-4fe1-8833-505cdfa99e1b\") "
Jul 19 14:59:00.993196 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.993169    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-config" (OuterVolumeSpecName: "console-config") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:59:00.993275 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.993250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-service-ca" (OuterVolumeSpecName: "service-ca") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:59:00.993327 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.993264    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:59:00.993327 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.993288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:59:00.995074 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.995046    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:59:00.995258 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.995240    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:59:00.995327 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:00.995262    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70308b9b-72bb-4fe1-8833-505cdfa99e1b-kube-api-access-hg8sd" (OuterVolumeSpecName: "kube-api-access-hg8sd") pod "70308b9b-72bb-4fe1-8833-505cdfa99e1b" (UID: "70308b9b-72bb-4fe1-8833-505cdfa99e1b"). InnerVolumeSpecName "kube-api-access-hg8sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:01.094002 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.093976    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-trusted-ca-bundle\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.094002 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.093998    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.094002 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.094007    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.094181 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.094018    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg8sd\" (UniqueName: \"kubernetes.io/projected/70308b9b-72bb-4fe1-8833-505cdfa99e1b-kube-api-access-hg8sd\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.094181 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.094026    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70308b9b-72bb-4fe1-8833-505cdfa99e1b-console-oauth-config\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.094181 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.094035    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-oauth-serving-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.094181 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.094043    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70308b9b-72bb-4fe1-8833-505cdfa99e1b-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:01.392858 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.392771    2577 generic.go:356] "Generic (PLEG): container finished" podID="70308b9b-72bb-4fe1-8833-505cdfa99e1b" containerID="197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3" exitCode=0
Jul 19 14:59:01.392858 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.392817    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579bf5847f-bljhq"
Jul 19 14:59:01.393033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.392856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579bf5847f-bljhq" event={"ID":"70308b9b-72bb-4fe1-8833-505cdfa99e1b","Type":"ContainerDied","Data":"197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3"}
Jul 19 14:59:01.393033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.392902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579bf5847f-bljhq" event={"ID":"70308b9b-72bb-4fe1-8833-505cdfa99e1b","Type":"ContainerDied","Data":"e993c19866de5302fe49e97cd92855826759e46c4ba23392438b4956255e2848"}
Jul 19 14:59:01.393033 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.392923    2577 scope.go:117] "RemoveContainer" containerID="197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3"
Jul 19 14:59:01.401418 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.401402    2577 scope.go:117] "RemoveContainer" containerID="197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3"
Jul 19 14:59:01.401730 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:59:01.401712    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3\": container with ID starting with 197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3 not found: ID does not exist" containerID="197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3"
Jul 19 14:59:01.401791 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.401738    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3"} err="failed to get container status \"197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3\": rpc error: code = NotFound desc = could not find container \"197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3\": container with ID starting with 197c5a9943a52d587c5d1d3568a3cf89b0cf5ad05787085c76b477d6210582f3 not found: ID does not exist"
Jul 19 14:59:01.417880 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.417860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-579bf5847f-bljhq"]
Jul 19 14:59:01.424434 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.424410    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-579bf5847f-bljhq"]
Jul 19 14:59:01.538568 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:01.538531    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70308b9b-72bb-4fe1-8833-505cdfa99e1b" path="/var/lib/kubelet/pods/70308b9b-72bb-4fe1-8833-505cdfa99e1b/volumes"
Jul 19 14:59:02.035022 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.034988    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"]
Jul 19 14:59:02.035452 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.035430    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a34cb1-83da-43ed-90b6-aba6fb0d4f14" containerName="manager"
Jul 19 14:59:02.035452 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.035448    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a34cb1-83da-43ed-90b6-aba6fb0d4f14" containerName="manager"
Jul 19 14:59:02.035606 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.035465    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70308b9b-72bb-4fe1-8833-505cdfa99e1b" containerName="console"
Jul 19 14:59:02.035606 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.035474    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="70308b9b-72bb-4fe1-8833-505cdfa99e1b" containerName="console"
Jul 19 14:59:02.035606 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.035597    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="70308b9b-72bb-4fe1-8833-505cdfa99e1b" containerName="console"
Jul 19 14:59:02.035746 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.035611    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a34cb1-83da-43ed-90b6-aba6fb0d4f14" containerName="manager"
Jul 19 14:59:02.039915 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.039894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.042351 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.042332    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hkhzm\""
Jul 19 14:59:02.050404 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.050381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"]
Jul 19 14:59:02.065763 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.065740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:02.202196 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202196 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d01314-3d84-44cc-b5c7-9cab54e14dea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202279    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.202400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.202387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr9vk\" (UniqueName: \"kubernetes.io/projected/11d01314-3d84-44cc-b5c7-9cab54e14dea-kube-api-access-vr9vk\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.302805 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.302805 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.302805 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303089 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr9vk\" (UniqueName: \"kubernetes.io/projected/11d01314-3d84-44cc-b5c7-9cab54e14dea-kube-api-access-vr9vk\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303089 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303089 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d01314-3d84-44cc-b5c7-9cab54e14dea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303089 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303089 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.302984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303335 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.303079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303335 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.303248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303335 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.303213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303490 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.303421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303582 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.303526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.303799 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.303780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d01314-3d84-44cc-b5c7-9cab54e14dea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.305514 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.305479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.305802 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.305785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.310741 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.310716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d01314-3d84-44cc-b5c7-9cab54e14dea-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.310958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.310941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr9vk\" (UniqueName: \"kubernetes.io/projected/11d01314-3d84-44cc-b5c7-9cab54e14dea-kube-api-access-vr9vk\") pod \"maas-default-gateway-openshift-default-56654dcf7-znlwk\" (UID: \"11d01314-3d84-44cc-b5c7-9cab54e14dea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.349919 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.349893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:02.391194 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.391156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:02.481813 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.481776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"]
Jul 19 14:59:02.484830 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:02.484793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d01314_3d84_44cc_b5c7_9cab54e14dea.slice/crio-f98706809def01e0c5fc424378144c6745ec6f17feb4dd02b3e65b06305339e2 WatchSource:0}: Error finding container f98706809def01e0c5fc424378144c6745ec6f17feb4dd02b3e65b06305339e2: Status 404 returned error can't find the container with id f98706809def01e0c5fc424378144c6745ec6f17feb4dd02b3e65b06305339e2
Jul 19 14:59:02.487117 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.487081    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:59:02.487200 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.487164    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:59:02.487239 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:02.487202    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 14:59:03.202423 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:03.202378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:03.300223 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:03.300195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:03.405180 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:03.405143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" event={"ID":"11d01314-3d84-44cc-b5c7-9cab54e14dea","Type":"ContainerStarted","Data":"3ea90fabda91b4aede4fc44c08bccb89c52a362d05af90053dc1be35e6853f73"}
Jul 19 14:59:03.405386 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:03.405191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" event={"ID":"11d01314-3d84-44cc-b5c7-9cab54e14dea","Type":"ContainerStarted","Data":"f98706809def01e0c5fc424378144c6745ec6f17feb4dd02b3e65b06305339e2"}
Jul 19 14:59:03.425623 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:03.425579    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" podStartSLOduration=1.425564198 podStartE2EDuration="1.425564198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:03.42397233 +0000 UTC m=+724.448981406" watchObservedRunningTime="2026-07-19 14:59:03.425564198 +0000 UTC m=+724.450573272"
Jul 19 14:59:04.350066 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.350030    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:04.354982 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.354950    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:04.409656 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.409627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:04.410569 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.410551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 14:59:04.488804 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.488773    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:04.489400 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.489370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:04.492334 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.492318    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.494771 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.494753    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 19 14:59:04.494837 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.494821    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wm2fr\""
Jul 19 14:59:04.500552 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.500525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:04.591287 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.591254    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:04.594438 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.594403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:04.625318 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.625248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmhfm\" (UniqueName: \"kubernetes.io/projected/3aa228e9-918a-43f2-bd4d-2837f9da84df-kube-api-access-lmhfm\") pod \"limitador-limitador-64c94fd64f-qvd5d\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.625445 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.625362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3aa228e9-918a-43f2-bd4d-2837f9da84df-config-file\") pod \"limitador-limitador-64c94fd64f-qvd5d\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.651412 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.651379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:04.726677 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.726637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3aa228e9-918a-43f2-bd4d-2837f9da84df-config-file\") pod \"limitador-limitador-64c94fd64f-qvd5d\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.726840 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.726691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmhfm\" (UniqueName: \"kubernetes.io/projected/3aa228e9-918a-43f2-bd4d-2837f9da84df-kube-api-access-lmhfm\") pod \"limitador-limitador-64c94fd64f-qvd5d\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.727285 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.727266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3aa228e9-918a-43f2-bd4d-2837f9da84df-config-file\") pod \"limitador-limitador-64c94fd64f-qvd5d\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.739246 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.739216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmhfm\" (UniqueName: \"kubernetes.io/projected/3aa228e9-918a-43f2-bd4d-2837f9da84df-kube-api-access-lmhfm\") pod \"limitador-limitador-64c94fd64f-qvd5d\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.802976 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.802944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:04.929403 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:04.929378    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:04.931506 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:04.931470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa228e9_918a_43f2_bd4d_2837f9da84df.slice/crio-e3c2a104c4e42d77d636700128fd0c4977e6bbd213208af72323d26129e365ae WatchSource:0}: Error finding container e3c2a104c4e42d77d636700128fd0c4977e6bbd213208af72323d26129e365ae: Status 404 returned error can't find the container with id e3c2a104c4e42d77d636700128fd0c4977e6bbd213208af72323d26129e365ae
Jul 19 14:59:05.292587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.292557    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-dzmbj"]
Jul 19 14:59:05.297455 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.297435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:05.299778 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.299759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x6fv7\""
Jul 19 14:59:05.302469 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.302450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dzmbj"]
Jul 19 14:59:05.328034 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.328009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:05.384299 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.384264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:05.416006 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.415975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d" event={"ID":"3aa228e9-918a-43f2-bd4d-2837f9da84df","Type":"ContainerStarted","Data":"e3c2a104c4e42d77d636700128fd0c4977e6bbd213208af72323d26129e365ae"}
Jul 19 14:59:05.432652 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.432622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwbzj\" (UniqueName: \"kubernetes.io/projected/f509b40a-c239-4d58-ab14-7bcac2090403-kube-api-access-kwbzj\") pod \"authorino-58cd955675-dzmbj\" (UID: \"f509b40a-c239-4d58-ab14-7bcac2090403\") " pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:05.505572 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.505542    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7zb7l"]
Jul 19 14:59:05.509031 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.509013    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:05.517370 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.517345    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7zb7l"]
Jul 19 14:59:05.534038 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.533996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwbzj\" (UniqueName: \"kubernetes.io/projected/f509b40a-c239-4d58-ab14-7bcac2090403-kube-api-access-kwbzj\") pod \"authorino-58cd955675-dzmbj\" (UID: \"f509b40a-c239-4d58-ab14-7bcac2090403\") " pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:05.543029 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.542942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwbzj\" (UniqueName: \"kubernetes.io/projected/f509b40a-c239-4d58-ab14-7bcac2090403-kube-api-access-kwbzj\") pod \"authorino-58cd955675-dzmbj\" (UID: \"f509b40a-c239-4d58-ab14-7bcac2090403\") " pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:05.608697 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.608568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:05.635649 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.634758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxxzz\" (UniqueName: \"kubernetes.io/projected/51cc1e6e-b40c-4901-984e-5b1150cc0044-kube-api-access-hxxzz\") pod \"authorino-75d556bd77-7zb7l\" (UID: \"51cc1e6e-b40c-4901-984e-5b1150cc0044\") " pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:05.736832 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.736709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxxzz\" (UniqueName: \"kubernetes.io/projected/51cc1e6e-b40c-4901-984e-5b1150cc0044-kube-api-access-hxxzz\") pod \"authorino-75d556bd77-7zb7l\" (UID: \"51cc1e6e-b40c-4901-984e-5b1150cc0044\") " pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:05.763229 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.763197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxxzz\" (UniqueName: \"kubernetes.io/projected/51cc1e6e-b40c-4901-984e-5b1150cc0044-kube-api-access-hxxzz\") pod \"authorino-75d556bd77-7zb7l\" (UID: \"51cc1e6e-b40c-4901-984e-5b1150cc0044\") " pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:05.816483 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.816294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dzmbj"]
Jul 19 14:59:05.817958 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.817933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:05.821313 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:05.821283    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf509b40a_c239_4d58_ab14_7bcac2090403.slice/crio-d741ad5d2a38277139781a7dfcbe281e6e23645eb817ba322cebb3fd956530bd WatchSource:0}: Error finding container d741ad5d2a38277139781a7dfcbe281e6e23645eb817ba322cebb3fd956530bd: Status 404 returned error can't find the container with id d741ad5d2a38277139781a7dfcbe281e6e23645eb817ba322cebb3fd956530bd
Jul 19 14:59:05.987272 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:05.987191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7zb7l"]
Jul 19 14:59:05.989520 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:05.989475    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51cc1e6e_b40c_4901_984e_5b1150cc0044.slice/crio-e4da8fdcdc2112488903bbbea25aec754fcba340a498a168f9a5e698fa8012f1 WatchSource:0}: Error finding container e4da8fdcdc2112488903bbbea25aec754fcba340a498a168f9a5e698fa8012f1: Status 404 returned error can't find the container with id e4da8fdcdc2112488903bbbea25aec754fcba340a498a168f9a5e698fa8012f1
Jul 19 14:59:06.422571 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:06.422529    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dzmbj" event={"ID":"f509b40a-c239-4d58-ab14-7bcac2090403","Type":"ContainerStarted","Data":"d741ad5d2a38277139781a7dfcbe281e6e23645eb817ba322cebb3fd956530bd"}
Jul 19 14:59:06.424005 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:06.423975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7zb7l" event={"ID":"51cc1e6e-b40c-4901-984e-5b1150cc0044","Type":"ContainerStarted","Data":"e4da8fdcdc2112488903bbbea25aec754fcba340a498a168f9a5e698fa8012f1"}
Jul 19 14:59:07.964690 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:07.964649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:08.068887 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:08.068852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:10.447490 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.447451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d" event={"ID":"3aa228e9-918a-43f2-bd4d-2837f9da84df","Type":"ContainerStarted","Data":"e1f8ef286fb89c6337d37c55e41ba235afe80a69ba47b8e6320be9063542c723"}
Jul 19 14:59:10.447939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.447567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:10.448865 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.448836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7zb7l" event={"ID":"51cc1e6e-b40c-4901-984e-5b1150cc0044","Type":"ContainerStarted","Data":"9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03"}
Jul 19 14:59:10.450139 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.450119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dzmbj" event={"ID":"f509b40a-c239-4d58-ab14-7bcac2090403","Type":"ContainerStarted","Data":"507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2"}
Jul 19 14:59:10.467050 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.467005    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d" podStartSLOduration=1.6196886510000001 podStartE2EDuration="6.4669922s" totalImagesPullingTime="4.847303549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:10.464660214 +0000 UTC m=+731.489669366" watchObservedRunningTime="2026-07-19 14:59:10.4669922 +0000 UTC m=+731.492001297"
Jul 19 14:59:10.479906 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.479865    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7zb7l" podStartSLOduration=1.623239498 podStartE2EDuration="5.479853009s" totalImagesPullingTime="3.856613511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:10.478709048 +0000 UTC m=+731.503723932" watchObservedRunningTime="2026-07-19 14:59:10.479853009 +0000 UTC m=+731.504862126"
Jul 19 14:59:10.492361 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.492320    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-dzmbj" podStartSLOduration=1.47844378 podStartE2EDuration="5.492307761s" totalImagesPullingTime="4.013863981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:10.491584072 +0000 UTC m=+731.516593148" watchObservedRunningTime="2026-07-19 14:59:10.492307761 +0000 UTC m=+731.517316875"
Jul 19 14:59:10.514334 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:10.514311    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dzmbj"]
Jul 19 14:59:12.458937 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:12.458901    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-dzmbj" podUID="f509b40a-c239-4d58-ab14-7bcac2090403" containerName="authorino" containerID="cri-o://507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2" gracePeriod=30
Jul 19 14:59:12.696048 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:12.696025    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:12.795658 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:12.795634    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwbzj\" (UniqueName: \"kubernetes.io/projected/f509b40a-c239-4d58-ab14-7bcac2090403-kube-api-access-kwbzj\") pod \"f509b40a-c239-4d58-ab14-7bcac2090403\" (UID: \"f509b40a-c239-4d58-ab14-7bcac2090403\") "
Jul 19 14:59:12.797424 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:12.797401    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f509b40a-c239-4d58-ab14-7bcac2090403-kube-api-access-kwbzj" (OuterVolumeSpecName: "kube-api-access-kwbzj") pod "f509b40a-c239-4d58-ab14-7bcac2090403" (UID: "f509b40a-c239-4d58-ab14-7bcac2090403"). InnerVolumeSpecName "kube-api-access-kwbzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:12.896143 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:12.896113    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwbzj\" (UniqueName: \"kubernetes.io/projected/f509b40a-c239-4d58-ab14-7bcac2090403-kube-api-access-kwbzj\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:13.464512 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.464458    2577 generic.go:356] "Generic (PLEG): container finished" podID="f509b40a-c239-4d58-ab14-7bcac2090403" containerID="507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2" exitCode=0
Jul 19 14:59:13.464512 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.464512    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dzmbj"
Jul 19 14:59:13.464996 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.464544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dzmbj" event={"ID":"f509b40a-c239-4d58-ab14-7bcac2090403","Type":"ContainerDied","Data":"507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2"}
Jul 19 14:59:13.464996 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.464587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dzmbj" event={"ID":"f509b40a-c239-4d58-ab14-7bcac2090403","Type":"ContainerDied","Data":"d741ad5d2a38277139781a7dfcbe281e6e23645eb817ba322cebb3fd956530bd"}
Jul 19 14:59:13.464996 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.464611    2577 scope.go:117] "RemoveContainer" containerID="507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2"
Jul 19 14:59:13.472995 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.472977    2577 scope.go:117] "RemoveContainer" containerID="507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2"
Jul 19 14:59:13.473238 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:59:13.473221    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2\": container with ID starting with 507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2 not found: ID does not exist" containerID="507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2"
Jul 19 14:59:13.473296 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.473245    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2"} err="failed to get container status \"507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2\": rpc error: code = NotFound desc = could not find container \"507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2\": container with ID starting with 507ec77d2b9513aa4625e49c040f42ac413b0b299f5053846ca061139aaecac2 not found: ID does not exist"
Jul 19 14:59:13.483734 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.483712    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dzmbj"]
Jul 19 14:59:13.489119 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.489098    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-dzmbj"]
Jul 19 14:59:13.539318 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:13.539296    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f509b40a-c239-4d58-ab14-7bcac2090403" path="/var/lib/kubelet/pods/f509b40a-c239-4d58-ab14-7bcac2090403/volumes"
Jul 19 14:59:21.455599 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:21.455567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:21.821767 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:21.821735    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:21.821934 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:21.821899    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d" podUID="3aa228e9-918a-43f2-bd4d-2837f9da84df" containerName="limitador" containerID="cri-o://e1f8ef286fb89c6337d37c55e41ba235afe80a69ba47b8e6320be9063542c723" gracePeriod=30
Jul 19 14:59:22.505517 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.505467    2577 generic.go:356] "Generic (PLEG): container finished" podID="3aa228e9-918a-43f2-bd4d-2837f9da84df" containerID="e1f8ef286fb89c6337d37c55e41ba235afe80a69ba47b8e6320be9063542c723" exitCode=0
Jul 19 14:59:22.505882 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.505536    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d" event={"ID":"3aa228e9-918a-43f2-bd4d-2837f9da84df","Type":"ContainerDied","Data":"e1f8ef286fb89c6337d37c55e41ba235afe80a69ba47b8e6320be9063542c723"}
Jul 19 14:59:22.765411 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.765354    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:22.765668 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.765644    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmhfm\" (UniqueName: \"kubernetes.io/projected/3aa228e9-918a-43f2-bd4d-2837f9da84df-kube-api-access-lmhfm\") pod \"3aa228e9-918a-43f2-bd4d-2837f9da84df\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") "
Jul 19 14:59:22.765766 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.765750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3aa228e9-918a-43f2-bd4d-2837f9da84df-config-file\") pod \"3aa228e9-918a-43f2-bd4d-2837f9da84df\" (UID: \"3aa228e9-918a-43f2-bd4d-2837f9da84df\") "
Jul 19 14:59:22.766213 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.766185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aa228e9-918a-43f2-bd4d-2837f9da84df-config-file" (OuterVolumeSpecName: "config-file") pod "3aa228e9-918a-43f2-bd4d-2837f9da84df" (UID: "3aa228e9-918a-43f2-bd4d-2837f9da84df"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:59:22.767457 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.767437    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa228e9-918a-43f2-bd4d-2837f9da84df-kube-api-access-lmhfm" (OuterVolumeSpecName: "kube-api-access-lmhfm") pod "3aa228e9-918a-43f2-bd4d-2837f9da84df" (UID: "3aa228e9-918a-43f2-bd4d-2837f9da84df"). InnerVolumeSpecName "kube-api-access-lmhfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:22.866812 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.866785    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3aa228e9-918a-43f2-bd4d-2837f9da84df-config-file\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:22.866812 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:22.866808    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmhfm\" (UniqueName: \"kubernetes.io/projected/3aa228e9-918a-43f2-bd4d-2837f9da84df-kube-api-access-lmhfm\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:23.511347 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:23.511318    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"
Jul 19 14:59:23.511804 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:23.511317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvd5d" event={"ID":"3aa228e9-918a-43f2-bd4d-2837f9da84df","Type":"ContainerDied","Data":"e3c2a104c4e42d77d636700128fd0c4977e6bbd213208af72323d26129e365ae"}
Jul 19 14:59:23.511804 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:23.511455    2577 scope.go:117] "RemoveContainer" containerID="e1f8ef286fb89c6337d37c55e41ba235afe80a69ba47b8e6320be9063542c723"
Jul 19 14:59:23.531534 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:23.531487    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:23.538220 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:23.538200    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvd5d"]
Jul 19 14:59:25.538527 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:25.538481    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa228e9-918a-43f2-bd4d-2837f9da84df" path="/var/lib/kubelet/pods/3aa228e9-918a-43f2-bd4d-2837f9da84df/volumes"
Jul 19 14:59:25.805397 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:25.805327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:25.903245 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:25.903218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:26.249402 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249367    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"]
Jul 19 14:59:26.249711 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249697    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa228e9-918a-43f2-bd4d-2837f9da84df" containerName="limitador"
Jul 19 14:59:26.249759 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249713    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa228e9-918a-43f2-bd4d-2837f9da84df" containerName="limitador"
Jul 19 14:59:26.249759 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249726    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f509b40a-c239-4d58-ab14-7bcac2090403" containerName="authorino"
Jul 19 14:59:26.249759 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249732    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f509b40a-c239-4d58-ab14-7bcac2090403" containerName="authorino"
Jul 19 14:59:26.249876 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249790    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f509b40a-c239-4d58-ab14-7bcac2090403" containerName="authorino"
Jul 19 14:59:26.249876 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.249800    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa228e9-918a-43f2-bd4d-2837f9da84df" containerName="limitador"
Jul 19 14:59:26.252811 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.252790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.255102 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.255080    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 14:59:26.255955 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.255924    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 19 14:59:26.255955 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.255951    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-sj7gf\""
Jul 19 14:59:26.256127 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.255981    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 19 14:59:26.261520 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.261473    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"]
Jul 19 14:59:26.286183 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.286140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:26.292369 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.292338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l4ql\" (UniqueName: \"kubernetes.io/projected/8cceab3a-9368-4286-8cca-e2c89e8ee996-kube-api-access-8l4ql\") pod \"postgres-69fb4594cf-82qjl\" (UID: \"8cceab3a-9368-4286-8cca-e2c89e8ee996\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.292458 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.292389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8cceab3a-9368-4286-8cca-e2c89e8ee996-data\") pod \"postgres-69fb4594cf-82qjl\" (UID: \"8cceab3a-9368-4286-8cca-e2c89e8ee996\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.393587 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.393541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8cceab3a-9368-4286-8cca-e2c89e8ee996-data\") pod \"postgres-69fb4594cf-82qjl\" (UID: \"8cceab3a-9368-4286-8cca-e2c89e8ee996\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.393758 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.393630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4ql\" (UniqueName: \"kubernetes.io/projected/8cceab3a-9368-4286-8cca-e2c89e8ee996-kube-api-access-8l4ql\") pod \"postgres-69fb4594cf-82qjl\" (UID: \"8cceab3a-9368-4286-8cca-e2c89e8ee996\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.393926 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.393908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8cceab3a-9368-4286-8cca-e2c89e8ee996-data\") pod \"postgres-69fb4594cf-82qjl\" (UID: \"8cceab3a-9368-4286-8cca-e2c89e8ee996\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.401698 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.401674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l4ql\" (UniqueName: \"kubernetes.io/projected/8cceab3a-9368-4286-8cca-e2c89e8ee996-kube-api-access-8l4ql\") pod \"postgres-69fb4594cf-82qjl\" (UID: \"8cceab3a-9368-4286-8cca-e2c89e8ee996\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.564442 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.564341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:26.696395 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:26.696368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"]
Jul 19 14:59:26.698531 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:26.698487    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cceab3a_9368_4286_8cca_e2c89e8ee996.slice/crio-9a2536f61624dce2207b6e7058e72144d501d7c020a3d2f61933ca32b7a7a5f4 WatchSource:0}: Error finding container 9a2536f61624dce2207b6e7058e72144d501d7c020a3d2f61933ca32b7a7a5f4: Status 404 returned error can't find the container with id 9a2536f61624dce2207b6e7058e72144d501d7c020a3d2f61933ca32b7a7a5f4
Jul 19 14:59:27.542650 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:27.542575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl" event={"ID":"8cceab3a-9368-4286-8cca-e2c89e8ee996","Type":"ContainerStarted","Data":"9a2536f61624dce2207b6e7058e72144d501d7c020a3d2f61933ca32b7a7a5f4"}
Jul 19 14:59:31.560044 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:31.559953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl" event={"ID":"8cceab3a-9368-4286-8cca-e2c89e8ee996","Type":"ContainerStarted","Data":"ababe6bb082575ba5eeb307c694d2efe518b969fe96ab6c4d9e6000154891ee2"}
Jul 19 14:59:31.560383 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:31.560055    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:31.576558 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:31.576513    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl" podStartSLOduration=1.037220253 podStartE2EDuration="5.576487972s" totalImagesPullingTime="4.539267719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:31.575260586 +0000 UTC m=+752.600269663" watchObservedRunningTime="2026-07-19 14:59:31.576487972 +0000 UTC m=+752.601497058"
Jul 19 14:59:37.594059 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:37.594029    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-82qjl"
Jul 19 14:59:37.933698 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:37.933594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:38.177921 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.177882    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-77fsm"]
Jul 19 14:59:38.181259 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.181242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:38.187976 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.187898    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-77fsm"]
Jul 19 14:59:38.295005 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.294961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrzlv\" (UniqueName: \"kubernetes.io/projected/24fe58cf-336f-4448-a100-453a13bb2499-kube-api-access-zrzlv\") pod \"authorino-5d4b67c446-77fsm\" (UID: \"24fe58cf-336f-4448-a100-453a13bb2499\") " pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:38.395994 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.395961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrzlv\" (UniqueName: \"kubernetes.io/projected/24fe58cf-336f-4448-a100-453a13bb2499-kube-api-access-zrzlv\") pod \"authorino-5d4b67c446-77fsm\" (UID: \"24fe58cf-336f-4448-a100-453a13bb2499\") " pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:38.398205 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.398172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:38.406239 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.406212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrzlv\" (UniqueName: \"kubernetes.io/projected/24fe58cf-336f-4448-a100-453a13bb2499-kube-api-access-zrzlv\") pod \"authorino-5d4b67c446-77fsm\" (UID: \"24fe58cf-336f-4448-a100-453a13bb2499\") " pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:38.433072 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.433044    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-77fsm"]
Jul 19 14:59:38.433261 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.433249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:38.458328 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.458296    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54b67cd97d-n7nt2"]
Jul 19 14:59:38.462472 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.462452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:38.471878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.471858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54b67cd97d-n7nt2"]
Jul 19 14:59:38.496462 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.496389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5zjj\" (UniqueName: \"kubernetes.io/projected/0c73ba36-521f-4d45-b995-649ddc798921-kube-api-access-f5zjj\") pod \"authorino-54b67cd97d-n7nt2\" (UID: \"0c73ba36-521f-4d45-b995-649ddc798921\") " pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:38.565985 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.565957    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-77fsm"]
Jul 19 14:59:38.568230 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:38.568204    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24fe58cf_336f_4448_a100_453a13bb2499.slice/crio-eff3507301aa6154fb6d2643d901166bc5122fdaa1f2f316b2c9142e8b74a86e WatchSource:0}: Error finding container eff3507301aa6154fb6d2643d901166bc5122fdaa1f2f316b2c9142e8b74a86e: Status 404 returned error can't find the container with id eff3507301aa6154fb6d2643d901166bc5122fdaa1f2f316b2c9142e8b74a86e
Jul 19 14:59:38.592850 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.592822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-77fsm" event={"ID":"24fe58cf-336f-4448-a100-453a13bb2499","Type":"ContainerStarted","Data":"eff3507301aa6154fb6d2643d901166bc5122fdaa1f2f316b2c9142e8b74a86e"}
Jul 19 14:59:38.597190 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.597169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5zjj\" (UniqueName: \"kubernetes.io/projected/0c73ba36-521f-4d45-b995-649ddc798921-kube-api-access-f5zjj\") pod \"authorino-54b67cd97d-n7nt2\" (UID: \"0c73ba36-521f-4d45-b995-649ddc798921\") " pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:38.604874 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.604855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5zjj\" (UniqueName: \"kubernetes.io/projected/0c73ba36-521f-4d45-b995-649ddc798921-kube-api-access-f5zjj\") pod \"authorino-54b67cd97d-n7nt2\" (UID: \"0c73ba36-521f-4d45-b995-649ddc798921\") " pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:38.702244 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.702185    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b67cd97d-n7nt2"]
Jul 19 14:59:38.702392 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.702381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:38.723395 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.723370    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bcf69bb66-h2d54"]
Jul 19 14:59:38.727823 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.727802    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:38.730256 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.730236    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 19 14:59:38.731939 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.731914    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bcf69bb66-h2d54"]
Jul 19 14:59:38.799006 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.798977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-tls-cert\") pod \"authorino-6bcf69bb66-h2d54\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") " pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:38.799151 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.799021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2kt9\" (UniqueName: \"kubernetes.io/projected/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-kube-api-access-s2kt9\") pod \"authorino-6bcf69bb66-h2d54\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") " pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:38.828003 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.827980    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b67cd97d-n7nt2"]
Jul 19 14:59:38.830016 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:38.829989    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c73ba36_521f_4d45_b995_649ddc798921.slice/crio-28bf042a01c01c5b312d2e62199acb62d3481075db1fffecf921ba742d310273 WatchSource:0}: Error finding container 28bf042a01c01c5b312d2e62199acb62d3481075db1fffecf921ba742d310273: Status 404 returned error can't find the container with id 28bf042a01c01c5b312d2e62199acb62d3481075db1fffecf921ba742d310273
Jul 19 14:59:38.899893 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.899867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-tls-cert\") pod \"authorino-6bcf69bb66-h2d54\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") " pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:38.899978 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.899905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kt9\" (UniqueName: \"kubernetes.io/projected/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-kube-api-access-s2kt9\") pod \"authorino-6bcf69bb66-h2d54\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") " pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:38.902281 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.902258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-tls-cert\") pod \"authorino-6bcf69bb66-h2d54\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") " pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:38.907711 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:38.907690    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kt9\" (UniqueName: \"kubernetes.io/projected/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-kube-api-access-s2kt9\") pod \"authorino-6bcf69bb66-h2d54\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") " pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:39.039136 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.039108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 14:59:39.187818 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.187760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bcf69bb66-h2d54"]
Jul 19 14:59:39.191198 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:39.191174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c5e3a00_e19b_4337_b6bf_4a0ed4849729.slice/crio-ffd8e475194c25582d887c027d36a9afe16f55a24678d8ac4ac0d01710f82ca8 WatchSource:0}: Error finding container ffd8e475194c25582d887c027d36a9afe16f55a24678d8ac4ac0d01710f82ca8: Status 404 returned error can't find the container with id ffd8e475194c25582d887c027d36a9afe16f55a24678d8ac4ac0d01710f82ca8
Jul 19 14:59:39.598700 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.598651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-77fsm" event={"ID":"24fe58cf-336f-4448-a100-453a13bb2499","Type":"ContainerStarted","Data":"bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca"}
Jul 19 14:59:39.599136 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.598682    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-77fsm" podUID="24fe58cf-336f-4448-a100-453a13bb2499" containerName="authorino" containerID="cri-o://bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca" gracePeriod=30
Jul 19 14:59:39.600563 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.600532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bcf69bb66-h2d54" event={"ID":"2c5e3a00-e19b-4337-b6bf-4a0ed4849729","Type":"ContainerStarted","Data":"b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331"}
Jul 19 14:59:39.600672 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.600577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bcf69bb66-h2d54" event={"ID":"2c5e3a00-e19b-4337-b6bf-4a0ed4849729","Type":"ContainerStarted","Data":"ffd8e475194c25582d887c027d36a9afe16f55a24678d8ac4ac0d01710f82ca8"}
Jul 19 14:59:39.603879 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.603211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b67cd97d-n7nt2" event={"ID":"0c73ba36-521f-4d45-b995-649ddc798921","Type":"ContainerStarted","Data":"06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21"}
Jul 19 14:59:39.603879 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.603246    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b67cd97d-n7nt2" event={"ID":"0c73ba36-521f-4d45-b995-649ddc798921","Type":"ContainerStarted","Data":"28bf042a01c01c5b312d2e62199acb62d3481075db1fffecf921ba742d310273"}
Jul 19 14:59:39.603879 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.603343    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-54b67cd97d-n7nt2" podUID="0c73ba36-521f-4d45-b995-649ddc798921" containerName="authorino" containerID="cri-o://06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21" gracePeriod=30
Jul 19 14:59:39.613408 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.613356    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-77fsm" podStartSLOduration=1.088928671 podStartE2EDuration="1.61333821s" totalImagesPullingTime="524.409539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:39.612351567 +0000 UTC m=+760.637360653" watchObservedRunningTime="2026-07-19 14:59:39.61333821 +0000 UTC m=+760.638347286"
Jul 19 14:59:39.628481 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.628428    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54b67cd97d-n7nt2" podStartSLOduration=1.2966245650000001 podStartE2EDuration="1.628410297s" totalImagesPullingTime="331.785732ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:39.626392529 +0000 UTC m=+760.651401605" watchObservedRunningTime="2026-07-19 14:59:39.628410297 +0000 UTC m=+760.653419373"
Jul 19 14:59:39.640225 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.640181    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bcf69bb66-h2d54" podStartSLOduration=1.359673975 podStartE2EDuration="1.64016466s" totalImagesPullingTime="280.490685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:39.639887954 +0000 UTC m=+760.664897031" watchObservedRunningTime="2026-07-19 14:59:39.64016466 +0000 UTC m=+760.665173737"
Jul 19 14:59:39.666048 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.666024    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7zb7l"]
Jul 19 14:59:39.666251 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.666230    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7zb7l" podUID="51cc1e6e-b40c-4901-984e-5b1150cc0044" containerName="authorino" containerID="cri-o://9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03" gracePeriod=30
Jul 19 14:59:39.894230 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.894205    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:39.943156 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.943134    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:39.953647 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:39.953628    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:40.017853 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.017824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrzlv\" (UniqueName: \"kubernetes.io/projected/24fe58cf-336f-4448-a100-453a13bb2499-kube-api-access-zrzlv\") pod \"24fe58cf-336f-4448-a100-453a13bb2499\" (UID: \"24fe58cf-336f-4448-a100-453a13bb2499\") "
Jul 19 14:59:40.017988 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.017908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5zjj\" (UniqueName: \"kubernetes.io/projected/0c73ba36-521f-4d45-b995-649ddc798921-kube-api-access-f5zjj\") pod \"0c73ba36-521f-4d45-b995-649ddc798921\" (UID: \"0c73ba36-521f-4d45-b995-649ddc798921\") "
Jul 19 14:59:40.017988 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.017957    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxxzz\" (UniqueName: \"kubernetes.io/projected/51cc1e6e-b40c-4901-984e-5b1150cc0044-kube-api-access-hxxzz\") pod \"51cc1e6e-b40c-4901-984e-5b1150cc0044\" (UID: \"51cc1e6e-b40c-4901-984e-5b1150cc0044\") "
Jul 19 14:59:40.019817 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.019787    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c73ba36-521f-4d45-b995-649ddc798921-kube-api-access-f5zjj" (OuterVolumeSpecName: "kube-api-access-f5zjj") pod "0c73ba36-521f-4d45-b995-649ddc798921" (UID: "0c73ba36-521f-4d45-b995-649ddc798921"). InnerVolumeSpecName "kube-api-access-f5zjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:40.019968 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.019830    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51cc1e6e-b40c-4901-984e-5b1150cc0044-kube-api-access-hxxzz" (OuterVolumeSpecName: "kube-api-access-hxxzz") pod "51cc1e6e-b40c-4901-984e-5b1150cc0044" (UID: "51cc1e6e-b40c-4901-984e-5b1150cc0044"). InnerVolumeSpecName "kube-api-access-hxxzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:40.019968 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.019867    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24fe58cf-336f-4448-a100-453a13bb2499-kube-api-access-zrzlv" (OuterVolumeSpecName: "kube-api-access-zrzlv") pod "24fe58cf-336f-4448-a100-453a13bb2499" (UID: "24fe58cf-336f-4448-a100-453a13bb2499"). InnerVolumeSpecName "kube-api-access-zrzlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:40.119162 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.119092    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrzlv\" (UniqueName: \"kubernetes.io/projected/24fe58cf-336f-4448-a100-453a13bb2499-kube-api-access-zrzlv\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:40.119162 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.119119    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5zjj\" (UniqueName: \"kubernetes.io/projected/0c73ba36-521f-4d45-b995-649ddc798921-kube-api-access-f5zjj\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:40.119162 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.119129    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxxzz\" (UniqueName: \"kubernetes.io/projected/51cc1e6e-b40c-4901-984e-5b1150cc0044-kube-api-access-hxxzz\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 14:59:40.613477 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.613440    2577 generic.go:356] "Generic (PLEG): container finished" podID="51cc1e6e-b40c-4901-984e-5b1150cc0044" containerID="9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03" exitCode=0
Jul 19 14:59:40.613941 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.613511    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7zb7l"
Jul 19 14:59:40.613941 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.613520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7zb7l" event={"ID":"51cc1e6e-b40c-4901-984e-5b1150cc0044","Type":"ContainerDied","Data":"9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03"}
Jul 19 14:59:40.613941 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.613560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7zb7l" event={"ID":"51cc1e6e-b40c-4901-984e-5b1150cc0044","Type":"ContainerDied","Data":"e4da8fdcdc2112488903bbbea25aec754fcba340a498a168f9a5e698fa8012f1"}
Jul 19 14:59:40.613941 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.613584    2577 scope.go:117] "RemoveContainer" containerID="9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03"
Jul 19 14:59:40.614888 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.614854    2577 generic.go:356] "Generic (PLEG): container finished" podID="24fe58cf-336f-4448-a100-453a13bb2499" containerID="bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca" exitCode=0
Jul 19 14:59:40.614995 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.614977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-77fsm" event={"ID":"24fe58cf-336f-4448-a100-453a13bb2499","Type":"ContainerDied","Data":"bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca"}
Jul 19 14:59:40.615056 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.615009    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-77fsm" event={"ID":"24fe58cf-336f-4448-a100-453a13bb2499","Type":"ContainerDied","Data":"eff3507301aa6154fb6d2643d901166bc5122fdaa1f2f316b2c9142e8b74a86e"}
Jul 19 14:59:40.615108 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.615071    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-77fsm"
Jul 19 14:59:40.616554 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.616529    2577 generic.go:356] "Generic (PLEG): container finished" podID="0c73ba36-521f-4d45-b995-649ddc798921" containerID="06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21" exitCode=0
Jul 19 14:59:40.616685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.616584    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b67cd97d-n7nt2"
Jul 19 14:59:40.616685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.616615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b67cd97d-n7nt2" event={"ID":"0c73ba36-521f-4d45-b995-649ddc798921","Type":"ContainerDied","Data":"06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21"}
Jul 19 14:59:40.616685 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.616649    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b67cd97d-n7nt2" event={"ID":"0c73ba36-521f-4d45-b995-649ddc798921","Type":"ContainerDied","Data":"28bf042a01c01c5b312d2e62199acb62d3481075db1fffecf921ba742d310273"}
Jul 19 14:59:40.623571 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.623469    2577 scope.go:117] "RemoveContainer" containerID="9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03"
Jul 19 14:59:40.623809 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:59:40.623785    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03\": container with ID starting with 9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03 not found: ID does not exist" containerID="9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03"
Jul 19 14:59:40.623878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.623818    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03"} err="failed to get container status \"9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03\": rpc error: code = NotFound desc = could not find container \"9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03\": container with ID starting with 9b6482917c605b1f23c58c5adbdae35d122ec506df8e51f7e2e51ac2724d7e03 not found: ID does not exist"
Jul 19 14:59:40.623878 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.623841    2577 scope.go:117] "RemoveContainer" containerID="bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca"
Jul 19 14:59:40.632116 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.632089    2577 scope.go:117] "RemoveContainer" containerID="bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca"
Jul 19 14:59:40.632333 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:59:40.632316    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca\": container with ID starting with bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca not found: ID does not exist" containerID="bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca"
Jul 19 14:59:40.632374 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.632339    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca"} err="failed to get container status \"bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca\": rpc error: code = NotFound desc = could not find container \"bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca\": container with ID starting with bfb14dd21422c888537e70c036751cde2bc3168f748b07508db5ba482efecaca not found: ID does not exist"
Jul 19 14:59:40.632374 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.632355    2577 scope.go:117] "RemoveContainer" containerID="06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21"
Jul 19 14:59:40.640556 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.640538    2577 scope.go:117] "RemoveContainer" containerID="06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21"
Jul 19 14:59:40.640805 ip-10-0-142-137 kubenswrapper[2577]: E0719 14:59:40.640786    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21\": container with ID starting with 06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21 not found: ID does not exist" containerID="06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21"
Jul 19 14:59:40.640855 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.640811    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21"} err="failed to get container status \"06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21\": rpc error: code = NotFound desc = could not find container \"06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21\": container with ID starting with 06494144b4c810922946ab293bad2dcefed180d1fec9e61ada99e1b9bc588e21 not found: ID does not exist"
Jul 19 14:59:40.641061 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.641045    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7zb7l"]
Jul 19 14:59:40.643389 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.643372    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7zb7l"]
Jul 19 14:59:40.652910 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.652889    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-77fsm"]
Jul 19 14:59:40.658390 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.658373    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-77fsm"]
Jul 19 14:59:40.667817 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.667768    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b67cd97d-n7nt2"]
Jul 19 14:59:40.671165 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:40.671142    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54b67cd97d-n7nt2"]
Jul 19 14:59:41.540338 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:41.540305    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c73ba36-521f-4d45-b995-649ddc798921" path="/var/lib/kubelet/pods/0c73ba36-521f-4d45-b995-649ddc798921/volumes"
Jul 19 14:59:41.540674 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:41.540660    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24fe58cf-336f-4448-a100-453a13bb2499" path="/var/lib/kubelet/pods/24fe58cf-336f-4448-a100-453a13bb2499/volumes"
Jul 19 14:59:41.540983 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:41.540971    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51cc1e6e-b40c-4901-984e-5b1150cc0044" path="/var/lib/kubelet/pods/51cc1e6e-b40c-4901-984e-5b1150cc0044/volumes"
Jul 19 14:59:42.834551 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:42.834490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:44.475630 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.475593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:44.568197 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.568163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:44.621501 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.621476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:44.660980 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.660950    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-fxkhv"]
Jul 19 14:59:44.661348 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661329    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51cc1e6e-b40c-4901-984e-5b1150cc0044" containerName="authorino"
Jul 19 14:59:44.661348 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661348    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="51cc1e6e-b40c-4901-984e-5b1150cc0044" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661358    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24fe58cf-336f-4448-a100-453a13bb2499" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661363    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="24fe58cf-336f-4448-a100-453a13bb2499" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661374    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c73ba36-521f-4d45-b995-649ddc798921" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661379    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c73ba36-521f-4d45-b995-649ddc798921" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661435    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="51cc1e6e-b40c-4901-984e-5b1150cc0044" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661442    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="24fe58cf-336f-4448-a100-453a13bb2499" containerName="authorino"
Jul 19 14:59:44.661509 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.661449    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c73ba36-521f-4d45-b995-649ddc798921" containerName="authorino"
Jul 19 14:59:44.665687 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.665670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.668252 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.668234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 19 14:59:44.668486 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.668469    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 19 14:59:44.668603 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.668469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hbc4m\""
Jul 19 14:59:44.674516 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.674465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-fxkhv"]
Jul 19 14:59:44.754625 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.754550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/751b788b-de44-44b1-9e2b-9a4c4b339349-webhook-cert\") pod \"maas-controller-74d45bc5b5-fxkhv\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") " pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.754625 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.754598    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxstp\" (UniqueName: \"kubernetes.io/projected/751b788b-de44-44b1-9e2b-9a4c4b339349-kube-api-access-lxstp\") pod \"maas-controller-74d45bc5b5-fxkhv\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") " pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.821519 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.821477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:44.855714 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.855689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxstp\" (UniqueName: \"kubernetes.io/projected/751b788b-de44-44b1-9e2b-9a4c4b339349-kube-api-access-lxstp\") pod \"maas-controller-74d45bc5b5-fxkhv\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") " pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.855848 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.855832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/751b788b-de44-44b1-9e2b-9a4c4b339349-webhook-cert\") pod \"maas-controller-74d45bc5b5-fxkhv\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") " pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.858125 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.858108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/751b788b-de44-44b1-9e2b-9a4c4b339349-webhook-cert\") pod \"maas-controller-74d45bc5b5-fxkhv\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") " pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.863175 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.863158    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxstp\" (UniqueName: \"kubernetes.io/projected/751b788b-de44-44b1-9e2b-9a4c4b339349-kube-api-access-lxstp\") pod \"maas-controller-74d45bc5b5-fxkhv\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") " pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:44.979337 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:44.979310    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:45.104919 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:45.104895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-fxkhv"]
Jul 19 14:59:45.106559 ip-10-0-142-137 kubenswrapper[2577]: W0719 14:59:45.106533    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751b788b_de44_44b1_9e2b_9a4c4b339349.slice/crio-4ea4b47d9c8866d0223e9238ea1554fa66275473079045c8910b7d73978afa0c WatchSource:0}: Error finding container 4ea4b47d9c8866d0223e9238ea1554fa66275473079045c8910b7d73978afa0c: Status 404 returned error can't find the container with id 4ea4b47d9c8866d0223e9238ea1554fa66275473079045c8910b7d73978afa0c
Jul 19 14:59:45.642752 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:45.642717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv" event={"ID":"751b788b-de44-44b1-9e2b-9a4c4b339349","Type":"ContainerStarted","Data":"4ea4b47d9c8866d0223e9238ea1554fa66275473079045c8910b7d73978afa0c"}
Jul 19 14:59:47.517003 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:47.516970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:47.634567 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:47.634470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:47.654369 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:47.654340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv" event={"ID":"751b788b-de44-44b1-9e2b-9a4c4b339349","Type":"ContainerStarted","Data":"88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e"}
Jul 19 14:59:47.654529 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:47.654458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 14:59:47.684396 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:47.684359    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv" podStartSLOduration=1.425172925 podStartE2EDuration="3.684340778s" totalImagesPullingTime="2.259167853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:47.682598287 +0000 UTC m=+768.707607363" watchObservedRunningTime="2026-07-19 14:59:47.684340778 +0000 UTC m=+768.709349853"
Jul 19 14:59:58.661591 ip-10-0-142-137 kubenswrapper[2577]: I0719 14:59:58.661561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 15:00:19.751737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:19.751688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:19.866787 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:19.866755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:20.412806 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.412768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:20.470480 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.470446    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"]
Jul 19 15:00:20.497030 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.496999    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"]
Jul 19 15:00:20.497182 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.497115    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.499793 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.499771    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 19 15:00:20.499925 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.499771    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scszj\""
Jul 19 15:00:20.545862 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.545829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxvmq\" (UniqueName: \"kubernetes.io/projected/fadfe2bc-8556-4b67-ae13-1340d4721e3d-kube-api-access-hxvmq\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.545996 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.545872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fadfe2bc-8556-4b67-ae13-1340d4721e3d-maas-api-tls\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.545996 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.545900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fadfe2bc-8556-4b67-ae13-1340d4721e3d-openshift-service-ca\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.575140 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.575110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:20.646339 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.646302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fadfe2bc-8556-4b67-ae13-1340d4721e3d-maas-api-tls\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.646530 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.646347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fadfe2bc-8556-4b67-ae13-1340d4721e3d-openshift-service-ca\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.646530 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.646411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxvmq\" (UniqueName: \"kubernetes.io/projected/fadfe2bc-8556-4b67-ae13-1340d4721e3d-kube-api-access-hxvmq\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.647061 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.647038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fadfe2bc-8556-4b67-ae13-1340d4721e3d-openshift-service-ca\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.648737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.648714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fadfe2bc-8556-4b67-ae13-1340d4721e3d-maas-api-tls\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.656808 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.656782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxvmq\" (UniqueName: \"kubernetes.io/projected/fadfe2bc-8556-4b67-ae13-1340d4721e3d-kube-api-access-hxvmq\") pod \"maas-api-857956d86b-6qlhb\" (UID: \"fadfe2bc-8556-4b67-ae13-1340d4721e3d\") " pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.807856 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.807821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:20.900998 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.900952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:20.960083 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.960052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"]
Jul 19 15:00:20.960672 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:00:20.960641    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfadfe2bc_8556_4b67_ae13_1340d4721e3d.slice/crio-ba40f4eb4e79b73fdee8dbb597b62ddbedebe5bb48ea28bded7750eaee65b9bc WatchSource:0}: Error finding container ba40f4eb4e79b73fdee8dbb597b62ddbedebe5bb48ea28bded7750eaee65b9bc: Status 404 returned error can't find the container with id ba40f4eb4e79b73fdee8dbb597b62ddbedebe5bb48ea28bded7750eaee65b9bc
Jul 19 15:00:20.999087 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:20.999055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:21.075440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.075361    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-56fl8"]
Jul 19 15:00:21.079963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.079945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.082439 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.082413    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 19 15:00:21.082599 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.082414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-htknr\""
Jul 19 15:00:21.088146 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.087932    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-56fl8"]
Jul 19 15:00:21.150453 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.150406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e1e882-27b2-45b6-8da0-868bdcc393bd-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-56fl8\" (UID: \"e9e1e882-27b2-45b6-8da0-868bdcc393bd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.150453 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.150455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqtxx\" (UniqueName: \"kubernetes.io/projected/e9e1e882-27b2-45b6-8da0-868bdcc393bd-kube-api-access-vqtxx\") pod \"payload-pre-processing-7677b774bb-56fl8\" (UID: \"e9e1e882-27b2-45b6-8da0-868bdcc393bd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.251482 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.251448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e1e882-27b2-45b6-8da0-868bdcc393bd-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-56fl8\" (UID: \"e9e1e882-27b2-45b6-8da0-868bdcc393bd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.251482 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.251490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqtxx\" (UniqueName: \"kubernetes.io/projected/e9e1e882-27b2-45b6-8da0-868bdcc393bd-kube-api-access-vqtxx\") pod \"payload-pre-processing-7677b774bb-56fl8\" (UID: \"e9e1e882-27b2-45b6-8da0-868bdcc393bd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.252123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.252102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e1e882-27b2-45b6-8da0-868bdcc393bd-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-56fl8\" (UID: \"e9e1e882-27b2-45b6-8da0-868bdcc393bd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.269287 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.269257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqtxx\" (UniqueName: \"kubernetes.io/projected/e9e1e882-27b2-45b6-8da0-868bdcc393bd-kube-api-access-vqtxx\") pod \"payload-pre-processing-7677b774bb-56fl8\" (UID: \"e9e1e882-27b2-45b6-8da0-868bdcc393bd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.392702 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.392593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:21.522944 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.522910    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-56fl8"]
Jul 19 15:00:21.525848 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:00:21.525816    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e1e882_27b2_45b6_8da0_868bdcc393bd.slice/crio-de1ac17bf02aa29cf2f1c04554da20660603222f7b9eca0793828316d09dd51e WatchSource:0}: Error finding container de1ac17bf02aa29cf2f1c04554da20660603222f7b9eca0793828316d09dd51e: Status 404 returned error can't find the container with id de1ac17bf02aa29cf2f1c04554da20660603222f7b9eca0793828316d09dd51e
Jul 19 15:00:21.806478 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.806441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb" event={"ID":"fadfe2bc-8556-4b67-ae13-1340d4721e3d","Type":"ContainerStarted","Data":"ba40f4eb4e79b73fdee8dbb597b62ddbedebe5bb48ea28bded7750eaee65b9bc"}
Jul 19 15:00:21.807527 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:21.807482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8" event={"ID":"e9e1e882-27b2-45b6-8da0-868bdcc393bd","Type":"ContainerStarted","Data":"de1ac17bf02aa29cf2f1c04554da20660603222f7b9eca0793828316d09dd51e"}
Jul 19 15:00:23.822105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:23.822006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8" event={"ID":"e9e1e882-27b2-45b6-8da0-868bdcc393bd","Type":"ContainerStarted","Data":"6fb55c0e188e1b596539c5b57ad227e39bcf9c2f478ea388168c53b551a785cb"}
Jul 19 15:00:23.822836 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:23.822809    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:00:23.829057 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:23.829024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb" event={"ID":"fadfe2bc-8556-4b67-ae13-1340d4721e3d","Type":"ContainerStarted","Data":"92e9999f65bb7b8bf9b6bfcae54dc198918552ad5de1f1a31c1b557d2ab6b247"}
Jul 19 15:00:23.829568 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:23.829546    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:23.860379 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:23.860328    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb" podStartSLOduration=1.336202775 podStartE2EDuration="3.860313891s" totalImagesPullingTime="2.524111116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:00:23.858698025 +0000 UTC m=+804.883707124" watchObservedRunningTime="2026-07-19 15:00:23.860313891 +0000 UTC m=+804.885322965"
Jul 19 15:00:23.860520 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:23.860417    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8" podStartSLOduration=0.825816714 podStartE2EDuration="2.860412686s" totalImagesPullingTime="2.034595972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:00:23.841636466 +0000 UTC m=+804.866645542" watchObservedRunningTime="2026-07-19 15:00:23.860412686 +0000 UTC m=+804.885421760"
Jul 19 15:00:28.854602 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:28.854531    2577 generic.go:356] "Generic (PLEG): container finished" podID="11d01314-3d84-44cc-b5c7-9cab54e14dea" containerID="3ea90fabda91b4aede4fc44c08bccb89c52a362d05af90053dc1be35e6853f73" exitCode=0
Jul 19 15:00:28.854988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:28.854601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" event={"ID":"11d01314-3d84-44cc-b5c7-9cab54e14dea","Type":"ContainerDied","Data":"3ea90fabda91b4aede4fc44c08bccb89c52a362d05af90053dc1be35e6853f73"}
Jul 19 15:00:28.854988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:28.854923    2577 scope.go:117] "RemoveContainer" containerID="3ea90fabda91b4aede4fc44c08bccb89c52a362d05af90053dc1be35e6853f73"
Jul 19 15:00:28.855823 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:28.855797    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:00:28.855911 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:28.855860    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:00:28.855911 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:28.855889    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:00:29.861375 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:29.861339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" event={"ID":"11d01314-3d84-44cc-b5c7-9cab54e14dea","Type":"ContainerStarted","Data":"e49a98ab146ec3f3240782c0ff8d971a95fb7a9e57ff883db17e0aa30bfe71a9"}
Jul 19 15:00:30.350951 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:30.350909    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 15:00:30.352032 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:30.352008    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-znlwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 19 15:00:30.352094 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:30.352056    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" podUID="11d01314-3d84-44cc-b5c7-9cab54e14dea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 19 15:00:30.846581 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:30.846554    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-857956d86b-6qlhb"
Jul 19 15:00:31.350559 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:31.350518    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-znlwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 19 15:00:31.350989 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:31.350577    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" podUID="11d01314-3d84-44cc-b5c7-9cab54e14dea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 19 15:00:31.884832 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:31.884799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:31.984859 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:31.984829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:32.350541 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:32.350474    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-znlwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 19 15:00:32.350729 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:32.350566    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" podUID="11d01314-3d84-44cc-b5c7-9cab54e14dea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 19 15:00:33.350877 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:33.350817    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-znlwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 19 15:00:33.351283 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:33.350910    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk" podUID="11d01314-3d84-44cc-b5c7-9cab54e14dea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 19 15:00:34.354076 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:34.354047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 15:00:34.354461 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:34.354296    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 15:00:34.354917 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:34.354902    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-znlwk"
Jul 19 15:00:35.841841 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:00:35.841814    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-56fl8"
Jul 19 15:01:01.812394 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:01.812361    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:01:07.269737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.269704    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"]
Jul 19 15:01:07.275022 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.275000    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.278366 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.278347    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 19 15:01:07.278467 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.278366    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bfqdt\""
Jul 19 15:01:07.278467 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.278350    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 19 15:01:07.278467 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.278347    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 19 15:01:07.282988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.282957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"]
Jul 19 15:01:07.389001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.388960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:01:07.458017 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.457985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.458017 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.458020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.458249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.458120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.458249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.458145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.458249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.458170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6k56\" (UniqueName: \"kubernetes.io/projected/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-kube-api-access-m6k56\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.458249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.458199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.558840 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.558755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.558840 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.558789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.558840 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.558836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.559139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.558863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.559139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.558977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6k56\" (UniqueName: \"kubernetes.io/projected/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-kube-api-access-m6k56\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.559139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.559020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.559342 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.559314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.559342 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.559331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.559459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.559386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.561108 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.561081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.561584 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.561562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.567765 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.567741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6k56\" (UniqueName: \"kubernetes.io/projected/6dd4e8c5-f078-4125-890c-2e1b4f5a529f-kube-api-access-m6k56\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v4gpb\" (UID: \"6dd4e8c5-f078-4125-890c-2e1b4f5a529f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.585484 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.585456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:07.716894 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:07.716710    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"]
Jul 19 15:01:07.720188 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:01:07.720160    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd4e8c5_f078_4125_890c_2e1b4f5a529f.slice/crio-ed9dbede691be027ab4779034638bc18d2dc320572aea048bb6db7a0c8b45744 WatchSource:0}: Error finding container ed9dbede691be027ab4779034638bc18d2dc320572aea048bb6db7a0c8b45744: Status 404 returned error can't find the container with id ed9dbede691be027ab4779034638bc18d2dc320572aea048bb6db7a0c8b45744
Jul 19 15:01:08.050467 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:08.050427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb" event={"ID":"6dd4e8c5-f078-4125-890c-2e1b4f5a529f","Type":"ContainerStarted","Data":"ed9dbede691be027ab4779034638bc18d2dc320572aea048bb6db7a0c8b45744"}
Jul 19 15:01:09.062702 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:09.056838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb" event={"ID":"6dd4e8c5-f078-4125-890c-2e1b4f5a529f","Type":"ContainerStarted","Data":"f62e63a73804a79c56339e74f75e222e669fd873236166cb0679201dbfac1711"}
Jul 19 15:01:09.075147 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:09.075051    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb" podStartSLOduration=0.972663779 podStartE2EDuration="2.075036576s" totalImagesPullingTime="1.102372797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:01:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:01:09.07424826 +0000 UTC m=+850.099257334" watchObservedRunningTime="2026-07-19 15:01:09.075036576 +0000 UTC m=+850.100045650"
Jul 19 15:01:17.586080 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:17.586041    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:17.586562 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:17.586101    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:17.598887 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:17.598857    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:18.110139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:18.110112    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v4gpb"
Jul 19 15:01:19.181689 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:01:19.181659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:02:02.177187 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.177145    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"]
Jul 19 15:02:02.180821 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.180799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.183443 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.183418    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 19 15:02:02.190114 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.190087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"]
Jul 19 15:02:02.211837 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.211793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:02:02.287210 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.287176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.287210 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.287210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f151b9-d71d-4739-88de-b61a0e96d3e3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.287415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.287284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.287415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.287322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.287415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.287359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.287415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.287398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2rxs\" (UniqueName: \"kubernetes.io/projected/b6f151b9-d71d-4739-88de-b61a0e96d3e3-kube-api-access-q2rxs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388490 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388490 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2rxs\" (UniqueName: \"kubernetes.io/projected/b6f151b9-d71d-4739-88de-b61a0e96d3e3-kube-api-access-q2rxs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388490 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388490 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f151b9-d71d-4739-88de-b61a0e96d3e3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388732 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388790 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.388841 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.388821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.390584 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.390563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f151b9-d71d-4739-88de-b61a0e96d3e3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.390873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.390855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f151b9-d71d-4739-88de-b61a0e96d3e3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.396056 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.396033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2rxs\" (UniqueName: \"kubernetes.io/projected/b6f151b9-d71d-4739-88de-b61a0e96d3e3-kube-api-access-q2rxs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft\" (UID: \"b6f151b9-d71d-4739-88de-b61a0e96d3e3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.493224 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.493191    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:02.620976 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:02.620948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"]
Jul 19 15:02:02.622674 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:02:02.622632    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f151b9_d71d_4739_88de_b61a0e96d3e3.slice/crio-b1c4f38a973d198290c89327d7d435e304c9303efc93fd691a1a30479647e32e WatchSource:0}: Error finding container b1c4f38a973d198290c89327d7d435e304c9303efc93fd691a1a30479647e32e: Status 404 returned error can't find the container with id b1c4f38a973d198290c89327d7d435e304c9303efc93fd691a1a30479647e32e
Jul 19 15:02:03.309108 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:03.309071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft" event={"ID":"b6f151b9-d71d-4739-88de-b61a0e96d3e3","Type":"ContainerStarted","Data":"0aada45dec2e4e9d92f0a32ce3c5877a67720a1671857956f277836a8db1dd68"}
Jul 19 15:02:03.309488 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:03.309131    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft" event={"ID":"b6f151b9-d71d-4739-88de-b61a0e96d3e3","Type":"ContainerStarted","Data":"b1c4f38a973d198290c89327d7d435e304c9303efc93fd691a1a30479647e32e"}
Jul 19 15:02:03.327373 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:03.327324    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft" podStartSLOduration=1.1563750210000001 podStartE2EDuration="1.327309304s" totalImagesPullingTime="170.934283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:02:03.32542696 +0000 UTC m=+904.350436034" watchObservedRunningTime="2026-07-19 15:02:03.327309304 +0000 UTC m=+904.352318378"
Jul 19 15:02:12.494073 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:12.494017    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:12.494641 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:12.494139    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:12.506750 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:12.506722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:13.366901 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:13.366874    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft"
Jul 19 15:02:25.906550 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:25.906467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:02:34.508197 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:34.508159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:02:38.288123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:38.288091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:02:55.048724 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:55.048674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:02:55.149358 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:02:55.149314    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:19.973967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:19.973927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:25.307979 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:25.307880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:25.364775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:25.364738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:33.635773 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.635736    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-595d697f79-vv4p9"]
Jul 19 15:03:33.639064 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.639043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.646750 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.646725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595d697f79-vv4p9"]
Jul 19 15:03:33.761269 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.761240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5398e1d7-d450-4c63-a321-4eb9025dd4a6-tls-cert\") pod \"authorino-595d697f79-vv4p9\" (UID: \"5398e1d7-d450-4c63-a321-4eb9025dd4a6\") " pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.761420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.761294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgkhz\" (UniqueName: \"kubernetes.io/projected/5398e1d7-d450-4c63-a321-4eb9025dd4a6-kube-api-access-sgkhz\") pod \"authorino-595d697f79-vv4p9\" (UID: \"5398e1d7-d450-4c63-a321-4eb9025dd4a6\") " pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.862728 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.862703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgkhz\" (UniqueName: \"kubernetes.io/projected/5398e1d7-d450-4c63-a321-4eb9025dd4a6-kube-api-access-sgkhz\") pod \"authorino-595d697f79-vv4p9\" (UID: \"5398e1d7-d450-4c63-a321-4eb9025dd4a6\") " pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.862852 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.862763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5398e1d7-d450-4c63-a321-4eb9025dd4a6-tls-cert\") pod \"authorino-595d697f79-vv4p9\" (UID: \"5398e1d7-d450-4c63-a321-4eb9025dd4a6\") " pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.865119 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.865098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5398e1d7-d450-4c63-a321-4eb9025dd4a6-tls-cert\") pod \"authorino-595d697f79-vv4p9\" (UID: \"5398e1d7-d450-4c63-a321-4eb9025dd4a6\") " pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.870941 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.870913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgkhz\" (UniqueName: \"kubernetes.io/projected/5398e1d7-d450-4c63-a321-4eb9025dd4a6-kube-api-access-sgkhz\") pod \"authorino-595d697f79-vv4p9\" (UID: \"5398e1d7-d450-4c63-a321-4eb9025dd4a6\") " pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:33.948704 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:33.948645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d697f79-vv4p9"
Jul 19 15:03:34.076576 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.076550    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595d697f79-vv4p9"]
Jul 19 15:03:34.078634 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:03:34.078605    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5398e1d7_d450_4c63_a321_4eb9025dd4a6.slice/crio-b66a0f30cf63326ea7320702c9e08997b3bf5eef2c7b40e03d318ce8929b01b1 WatchSource:0}: Error finding container b66a0f30cf63326ea7320702c9e08997b3bf5eef2c7b40e03d318ce8929b01b1: Status 404 returned error can't find the container with id b66a0f30cf63326ea7320702c9e08997b3bf5eef2c7b40e03d318ce8929b01b1
Jul 19 15:03:34.079938 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.079914    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:03:34.737599 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.737558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d697f79-vv4p9" event={"ID":"5398e1d7-d450-4c63-a321-4eb9025dd4a6","Type":"ContainerStarted","Data":"e14ef9146cba58ed49699d04cf0ca5bdfe3b3eb004c420f6c61fae2c2d9bc0c7"}
Jul 19 15:03:34.737986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.737609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d697f79-vv4p9" event={"ID":"5398e1d7-d450-4c63-a321-4eb9025dd4a6","Type":"ContainerStarted","Data":"b66a0f30cf63326ea7320702c9e08997b3bf5eef2c7b40e03d318ce8929b01b1"}
Jul 19 15:03:34.753616 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.753575    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-595d697f79-vv4p9" podStartSLOduration=1.2955971640000001 podStartE2EDuration="1.75356246s" totalImagesPullingTime="457.965296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:03:34.753116734 +0000 UTC m=+995.778125831" watchObservedRunningTime="2026-07-19 15:03:34.75356246 +0000 UTC m=+995.778571560"
Jul 19 15:03:34.788361 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.788275    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bcf69bb66-h2d54"]
Jul 19 15:03:34.788798 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:34.788744    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6bcf69bb66-h2d54" podUID="2c5e3a00-e19b-4337-b6bf-4a0ed4849729" containerName="authorino" containerID="cri-o://b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331" gracePeriod=30
Jul 19 15:03:35.064896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.064869    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 15:03:35.173233 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.173200    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2kt9\" (UniqueName: \"kubernetes.io/projected/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-kube-api-access-s2kt9\") pod \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") "
Jul 19 15:03:35.173407 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.173275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-tls-cert\") pod \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\" (UID: \"2c5e3a00-e19b-4337-b6bf-4a0ed4849729\") "
Jul 19 15:03:35.175260 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.175232    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-kube-api-access-s2kt9" (OuterVolumeSpecName: "kube-api-access-s2kt9") pod "2c5e3a00-e19b-4337-b6bf-4a0ed4849729" (UID: "2c5e3a00-e19b-4337-b6bf-4a0ed4849729"). InnerVolumeSpecName "kube-api-access-s2kt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:03:35.184265 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.184226    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2c5e3a00-e19b-4337-b6bf-4a0ed4849729" (UID: "2c5e3a00-e19b-4337-b6bf-4a0ed4849729"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:03:35.274229 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.274186    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2kt9\" (UniqueName: \"kubernetes.io/projected/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-kube-api-access-s2kt9\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:03:35.274229 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.274215    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c5e3a00-e19b-4337-b6bf-4a0ed4849729-tls-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:03:35.312685 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.312584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:35.415852 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.415822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:35.744533 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.744475    2577 generic.go:356] "Generic (PLEG): container finished" podID="2c5e3a00-e19b-4337-b6bf-4a0ed4849729" containerID="b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331" exitCode=0
Jul 19 15:03:35.744987 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.744528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bcf69bb66-h2d54" event={"ID":"2c5e3a00-e19b-4337-b6bf-4a0ed4849729","Type":"ContainerDied","Data":"b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331"}
Jul 19 15:03:35.744987 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.744583    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bcf69bb66-h2d54" event={"ID":"2c5e3a00-e19b-4337-b6bf-4a0ed4849729","Type":"ContainerDied","Data":"ffd8e475194c25582d887c027d36a9afe16f55a24678d8ac4ac0d01710f82ca8"}
Jul 19 15:03:35.744987 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.744608    2577 scope.go:117] "RemoveContainer" containerID="b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331"
Jul 19 15:03:35.744987 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.744538    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bcf69bb66-h2d54"
Jul 19 15:03:35.754308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.754290    2577 scope.go:117] "RemoveContainer" containerID="b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331"
Jul 19 15:03:35.754619 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:03:35.754601    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331\": container with ID starting with b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331 not found: ID does not exist" containerID="b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331"
Jul 19 15:03:35.754699 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.754626    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331"} err="failed to get container status \"b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331\": rpc error: code = NotFound desc = could not find container \"b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331\": container with ID starting with b5525f24b8b5183fad6518868f1d985aa2893e1d98af74ec843b4783f9863331 not found: ID does not exist"
Jul 19 15:03:35.808710 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.808662    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bcf69bb66-h2d54"]
Jul 19 15:03:35.812195 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:35.812170    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bcf69bb66-h2d54"]
Jul 19 15:03:36.568817 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:36.568758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:36.667383 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:36.667332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:37.539692 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:37.539661    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c5e3a00-e19b-4337-b6bf-4a0ed4849729" path="/var/lib/kubelet/pods/2c5e3a00-e19b-4337-b6bf-4a0ed4849729/volumes"
Jul 19 15:03:54.004207 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:03:54.004146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:04:23.443099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.443058    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-fxkhv"]
Jul 19 15:04:23.443685 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.443606    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv" podUID="751b788b-de44-44b1-9e2b-9a4c4b339349" containerName="manager" containerID="cri-o://88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e" gracePeriod=30
Jul 19 15:04:23.695013 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.694941    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 15:04:23.781197 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.781161    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/751b788b-de44-44b1-9e2b-9a4c4b339349-webhook-cert\") pod \"751b788b-de44-44b1-9e2b-9a4c4b339349\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") "
Jul 19 15:04:23.781364 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.781309    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxstp\" (UniqueName: \"kubernetes.io/projected/751b788b-de44-44b1-9e2b-9a4c4b339349-kube-api-access-lxstp\") pod \"751b788b-de44-44b1-9e2b-9a4c4b339349\" (UID: \"751b788b-de44-44b1-9e2b-9a4c4b339349\") "
Jul 19 15:04:23.783431 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.783403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/751b788b-de44-44b1-9e2b-9a4c4b339349-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "751b788b-de44-44b1-9e2b-9a4c4b339349" (UID: "751b788b-de44-44b1-9e2b-9a4c4b339349"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:04:23.783583 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.783427    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751b788b-de44-44b1-9e2b-9a4c4b339349-kube-api-access-lxstp" (OuterVolumeSpecName: "kube-api-access-lxstp") pod "751b788b-de44-44b1-9e2b-9a4c4b339349" (UID: "751b788b-de44-44b1-9e2b-9a4c4b339349"). InnerVolumeSpecName "kube-api-access-lxstp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:04:23.882365 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.882334    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxstp\" (UniqueName: \"kubernetes.io/projected/751b788b-de44-44b1-9e2b-9a4c4b339349-kube-api-access-lxstp\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:04:23.882365 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.882361    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/751b788b-de44-44b1-9e2b-9a4c4b339349-webhook-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:04:23.972800 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.972726    2577 generic.go:356] "Generic (PLEG): container finished" podID="751b788b-de44-44b1-9e2b-9a4c4b339349" containerID="88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e" exitCode=0
Jul 19 15:04:23.972800 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.972776    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv"
Jul 19 15:04:23.972968 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.972811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv" event={"ID":"751b788b-de44-44b1-9e2b-9a4c4b339349","Type":"ContainerDied","Data":"88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e"}
Jul 19 15:04:23.972968 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.972867    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-fxkhv" event={"ID":"751b788b-de44-44b1-9e2b-9a4c4b339349","Type":"ContainerDied","Data":"4ea4b47d9c8866d0223e9238ea1554fa66275473079045c8910b7d73978afa0c"}
Jul 19 15:04:23.972968 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.972895    2577 scope.go:117] "RemoveContainer" containerID="88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e"
Jul 19 15:04:23.983631 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.983613    2577 scope.go:117] "RemoveContainer" containerID="88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e"
Jul 19 15:04:23.983873 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:04:23.983858    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e\": container with ID starting with 88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e not found: ID does not exist" containerID="88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e"
Jul 19 15:04:23.983923 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.983881    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e"} err="failed to get container status \"88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e\": rpc error: code = NotFound desc = could not find container \"88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e\": container with ID starting with 88d47450824cd7fdb0b2e807230675f2eb0833d1f5ae532af0fa7a4d34f2458e not found: ID does not exist"
Jul 19 15:04:24.001129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:23.997439    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-fxkhv"]
Jul 19 15:04:24.003559 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:24.003534    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-fxkhv"]
Jul 19 15:04:25.538949 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:25.538917    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="751b788b-de44-44b1-9e2b-9a4c4b339349" path="/var/lib/kubelet/pods/751b788b-de44-44b1-9e2b-9a4c4b339349/volumes"
Jul 19 15:04:58.202735 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:04:58.202653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:05:24.826868 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.826830    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-p97jb"]
Jul 19 15:05:24.827284 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.827203    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c5e3a00-e19b-4337-b6bf-4a0ed4849729" containerName="authorino"
Jul 19 15:05:24.827284 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.827215    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c5e3a00-e19b-4337-b6bf-4a0ed4849729" containerName="authorino"
Jul 19 15:05:24.827284 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.827222    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751b788b-de44-44b1-9e2b-9a4c4b339349" containerName="manager"
Jul 19 15:05:24.827284 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.827228    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="751b788b-de44-44b1-9e2b-9a4c4b339349" containerName="manager"
Jul 19 15:05:24.827284 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.827280    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c5e3a00-e19b-4337-b6bf-4a0ed4849729" containerName="authorino"
Jul 19 15:05:24.827446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.827289    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="751b788b-de44-44b1-9e2b-9a4c4b339349" containerName="manager"
Jul 19 15:05:24.831114 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.831089    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:24.834442 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.834418    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 19 15:05:24.834575 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.834468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 19 15:05:24.834575 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.834467    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hbc4m\""
Jul 19 15:05:24.840371 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.840347    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-p97jb"]
Jul 19 15:05:24.897031 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.896997    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhw5\" (UniqueName: \"kubernetes.io/projected/cdd2ad0d-0428-4b5a-a192-36aa15e61664-kube-api-access-nvhw5\") pod \"maas-controller-74d45bc5b5-p97jb\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") " pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:24.897031 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.897031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd2ad0d-0428-4b5a-a192-36aa15e61664-webhook-cert\") pod \"maas-controller-74d45bc5b5-p97jb\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") " pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:24.997479 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.997444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhw5\" (UniqueName: \"kubernetes.io/projected/cdd2ad0d-0428-4b5a-a192-36aa15e61664-kube-api-access-nvhw5\") pod \"maas-controller-74d45bc5b5-p97jb\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") " pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:24.997479 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.997481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd2ad0d-0428-4b5a-a192-36aa15e61664-webhook-cert\") pod \"maas-controller-74d45bc5b5-p97jb\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") " pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:24.999965 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:24.999943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd2ad0d-0428-4b5a-a192-36aa15e61664-webhook-cert\") pod \"maas-controller-74d45bc5b5-p97jb\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") " pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:25.005036 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:25.005013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhw5\" (UniqueName: \"kubernetes.io/projected/cdd2ad0d-0428-4b5a-a192-36aa15e61664-kube-api-access-nvhw5\") pod \"maas-controller-74d45bc5b5-p97jb\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") " pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:25.142416 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:25.142328    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:25.281005 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:25.280971    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-p97jb"]
Jul 19 15:05:26.277137 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:26.277092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-p97jb" event={"ID":"cdd2ad0d-0428-4b5a-a192-36aa15e61664","Type":"ContainerStarted","Data":"c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03"}
Jul 19 15:05:26.277546 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:26.277146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-p97jb" event={"ID":"cdd2ad0d-0428-4b5a-a192-36aa15e61664","Type":"ContainerStarted","Data":"01d257619c1e85eb9a7f8a46426363a656c931101dfa739a5f6e15cb62259444"}
Jul 19 15:05:26.277546 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:26.277217    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:26.293510 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:26.293447    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74d45bc5b5-p97jb" podStartSLOduration=1.732359721 podStartE2EDuration="2.293431s" totalImagesPullingTime="561.071279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:05:26.292337539 +0000 UTC m=+1107.317346615" watchObservedRunningTime="2026-07-19 15:05:26.293431 +0000 UTC m=+1107.318440076"
Jul 19 15:05:37.284108 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:37.284077    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:41.859018 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:41.858984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:05:41.969816 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:41.969788    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-p97jb"]
Jul 19 15:05:41.970000 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:41.969968    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74d45bc5b5-p97jb" podUID="cdd2ad0d-0428-4b5a-a192-36aa15e61664" containerName="manager" containerID="cri-o://c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03" gracePeriod=30
Jul 19 15:05:42.222439 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.222417    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:42.350631 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.350603    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvhw5\" (UniqueName: \"kubernetes.io/projected/cdd2ad0d-0428-4b5a-a192-36aa15e61664-kube-api-access-nvhw5\") pod \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") "
Jul 19 15:05:42.350789 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.350663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd2ad0d-0428-4b5a-a192-36aa15e61664-webhook-cert\") pod \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\" (UID: \"cdd2ad0d-0428-4b5a-a192-36aa15e61664\") "
Jul 19 15:05:42.352700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.352673    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd2ad0d-0428-4b5a-a192-36aa15e61664-kube-api-access-nvhw5" (OuterVolumeSpecName: "kube-api-access-nvhw5") pod "cdd2ad0d-0428-4b5a-a192-36aa15e61664" (UID: "cdd2ad0d-0428-4b5a-a192-36aa15e61664"). InnerVolumeSpecName "kube-api-access-nvhw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:05:42.352823 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.352677    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd2ad0d-0428-4b5a-a192-36aa15e61664-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cdd2ad0d-0428-4b5a-a192-36aa15e61664" (UID: "cdd2ad0d-0428-4b5a-a192-36aa15e61664"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:05:42.357277 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.357254    2577 generic.go:356] "Generic (PLEG): container finished" podID="cdd2ad0d-0428-4b5a-a192-36aa15e61664" containerID="c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03" exitCode=0
Jul 19 15:05:42.357382 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.357298    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-p97jb"
Jul 19 15:05:42.357382 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.357347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-p97jb" event={"ID":"cdd2ad0d-0428-4b5a-a192-36aa15e61664","Type":"ContainerDied","Data":"c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03"}
Jul 19 15:05:42.357459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.357409    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-p97jb" event={"ID":"cdd2ad0d-0428-4b5a-a192-36aa15e61664","Type":"ContainerDied","Data":"01d257619c1e85eb9a7f8a46426363a656c931101dfa739a5f6e15cb62259444"}
Jul 19 15:05:42.357459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.357439    2577 scope.go:117] "RemoveContainer" containerID="c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03"
Jul 19 15:05:42.366706 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.366691    2577 scope.go:117] "RemoveContainer" containerID="c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03"
Jul 19 15:05:42.366955 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:05:42.366936    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03\": container with ID starting with c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03 not found: ID does not exist" containerID="c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03"
Jul 19 15:05:42.367013 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.366964    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03"} err="failed to get container status \"c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03\": rpc error: code = NotFound desc = could not find container \"c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03\": container with ID starting with c14bf50a1feea79adcf987108261fd275e417ee56cfa85768f1824ec080a0c03 not found: ID does not exist"
Jul 19 15:05:42.378567 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.378546    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-p97jb"]
Jul 19 15:05:42.382038 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.382019    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-p97jb"]
Jul 19 15:05:42.452011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.451954    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvhw5\" (UniqueName: \"kubernetes.io/projected/cdd2ad0d-0428-4b5a-a192-36aa15e61664-kube-api-access-nvhw5\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:05:42.452011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:42.451974    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd2ad0d-0428-4b5a-a192-36aa15e61664-webhook-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:05:43.084559 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.084530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:05:43.200828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.200797    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-2kh6h"]
Jul 19 15:05:43.201143 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.201131    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd2ad0d-0428-4b5a-a192-36aa15e61664" containerName="manager"
Jul 19 15:05:43.201202 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.201145    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd2ad0d-0428-4b5a-a192-36aa15e61664" containerName="manager"
Jul 19 15:05:43.201202 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.201201    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd2ad0d-0428-4b5a-a192-36aa15e61664" containerName="manager"
Jul 19 15:05:43.205554 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.205534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.209176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.209157    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 19 15:05:43.209176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.209172    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 19 15:05:43.209328 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.209174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hbc4m\""
Jul 19 15:05:43.214142 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.214116    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-2kh6h"]
Jul 19 15:05:43.359777 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.359688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58-webhook-cert\") pod \"maas-controller-74d45bc5b5-2kh6h\" (UID: \"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58\") " pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.359777 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.359737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5v6k\" (UniqueName: \"kubernetes.io/projected/2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58-kube-api-access-t5v6k\") pod \"maas-controller-74d45bc5b5-2kh6h\" (UID: \"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58\") " pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.460543 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.460513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58-webhook-cert\") pod \"maas-controller-74d45bc5b5-2kh6h\" (UID: \"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58\") " pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.460679 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.460548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5v6k\" (UniqueName: \"kubernetes.io/projected/2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58-kube-api-access-t5v6k\") pod \"maas-controller-74d45bc5b5-2kh6h\" (UID: \"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58\") " pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.462910 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.462878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58-webhook-cert\") pod \"maas-controller-74d45bc5b5-2kh6h\" (UID: \"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58\") " pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.468314 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.468285    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5v6k\" (UniqueName: \"kubernetes.io/projected/2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58-kube-api-access-t5v6k\") pod \"maas-controller-74d45bc5b5-2kh6h\" (UID: \"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58\") " pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.517136 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.517113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:43.542119 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.542086    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd2ad0d-0428-4b5a-a192-36aa15e61664" path="/var/lib/kubelet/pods/cdd2ad0d-0428-4b5a-a192-36aa15e61664/volumes"
Jul 19 15:05:43.645111 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:43.644959    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d45bc5b5-2kh6h"]
Jul 19 15:05:43.647962 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:05:43.647937    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2eb6f1_4357_4b7a_b91f_2a906bcc8f58.slice/crio-f19faed297f53f7af37b08018a8fd9fc7988a35e37118c272336867bd18503ee WatchSource:0}: Error finding container f19faed297f53f7af37b08018a8fd9fc7988a35e37118c272336867bd18503ee: Status 404 returned error can't find the container with id f19faed297f53f7af37b08018a8fd9fc7988a35e37118c272336867bd18503ee
Jul 19 15:05:44.369569 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:44.369533    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h" event={"ID":"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58","Type":"ContainerStarted","Data":"80b3c8aa8550fe8801a46285133e01568bf14f94c27e299caff91b6c357d5865"}
Jul 19 15:05:44.369960 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:44.369591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h" event={"ID":"2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58","Type":"ContainerStarted","Data":"f19faed297f53f7af37b08018a8fd9fc7988a35e37118c272336867bd18503ee"}
Jul 19 15:05:44.369960 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:44.369613    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:05:44.387059 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:44.387012    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h" podStartSLOduration=1.077206596 podStartE2EDuration="1.386998841s" totalImagesPullingTime="309.792245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:05:44.384481884 +0000 UTC m=+1125.409490952" watchObservedRunningTime="2026-07-19 15:05:44.386998841 +0000 UTC m=+1125.412007907"
Jul 19 15:05:52.551566 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:52.551529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:05:55.376066 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:05:55.376030    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74d45bc5b5-2kh6h"
Jul 19 15:06:21.969513 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:21.969415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:22.072388 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:22.072354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:22.198384 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:22.198347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:27.878373 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:27.878341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:27.939458 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:27.939425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:28.019248 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:28.019213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:28.117980 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:28.117949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:28.251092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:28.251061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:33.877813 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:33.877781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:33.943158 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:06:33.943131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:33.347922 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:08:33.347891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:34.677415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:08:34.677387    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:10.920285 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:10:10.920248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:30.499671 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:11:30.499566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:41.610874 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:11:41.610840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:41.711208 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:11:41.711176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:17.656236 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:17.656163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:33.654239 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.654204    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"]
Jul 19 15:14:33.658324 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.658299    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.660605 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.660582    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-79056b85-kserve-self-signed-certs\""
Jul 19 15:14:33.667351 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.667328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"]
Jul 19 15:14:33.689557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.689530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:33.821567 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.821527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxvfl\" (UniqueName: \"kubernetes.io/projected/7b45e4c9-c988-40c5-a304-36c4e22f515b-kube-api-access-rxvfl\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.821757 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.821576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-model-cache\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.821757 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.821714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-tmp-dir\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.821875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.821766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b45e4c9-c988-40c5-a304-36c4e22f515b-tls-certs\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.821875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.821829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-dshm\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.821875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.821864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-home\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923073 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-tmp-dir\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923073 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b45e4c9-c988-40c5-a304-36c4e22f515b-tls-certs\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923073 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-dshm\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923339 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-home\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923339 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxvfl\" (UniqueName: \"kubernetes.io/projected/7b45e4c9-c988-40c5-a304-36c4e22f515b-kube-api-access-rxvfl\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923339 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-model-cache\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923529 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-tmp-dir\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923587 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-home\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.923627 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.923593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-model-cache\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.925387 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.925370    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-dshm\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.925562 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.925546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b45e4c9-c988-40c5-a304-36c4e22f515b-tls-certs\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.930432 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.930409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxvfl\" (UniqueName: \"kubernetes.io/projected/7b45e4c9-c988-40c5-a304-36c4e22f515b-kube-api-access-rxvfl\") pod \"e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") " pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:33.968896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:33.968871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:34.099896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:34.099874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"]
Jul 19 15:14:34.101941 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:14:34.101906    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b45e4c9_c988_40c5_a304_36c4e22f515b.slice/crio-deef0b160e9965a22c3c1a538b0cda335666b8ad71885334e4c3a78adb693732 WatchSource:0}: Error finding container deef0b160e9965a22c3c1a538b0cda335666b8ad71885334e4c3a78adb693732: Status 404 returned error can't find the container with id deef0b160e9965a22c3c1a538b0cda335666b8ad71885334e4c3a78adb693732
Jul 19 15:14:34.825559 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:34.825516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7" event={"ID":"7b45e4c9-c988-40c5-a304-36c4e22f515b","Type":"ContainerStarted","Data":"b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa"}
Jul 19 15:14:34.825967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:34.825567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7" event={"ID":"7b45e4c9-c988-40c5-a304-36c4e22f515b","Type":"ContainerStarted","Data":"deef0b160e9965a22c3c1a538b0cda335666b8ad71885334e4c3a78adb693732"}
Jul 19 15:14:34.841238 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:34.841186    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7" podStartSLOduration=1.841168304 podStartE2EDuration="1.841168304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:14:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:14:34.840769201 +0000 UTC m=+1655.865778275" watchObservedRunningTime="2026-07-19 15:14:34.841168304 +0000 UTC m=+1655.866177381"
Jul 19 15:14:43.969898 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:43.969862    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:43.970298 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:43.969948    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:43.981977 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:43.981960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:14:44.882155 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:14:44.882123    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:15:00.145661 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.145631    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"]
Jul 19 15:15:00.149285 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.149266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"
Jul 19 15:15:00.151696 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.151672    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:15:00.155331 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.155309    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"]
Jul 19 15:15:00.161229 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.161209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"
Jul 19 15:15:00.498205 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.498067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"]
Jul 19 15:15:00.500344 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:15:00.500318    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84bfcad4_da42_4127_992d_d2f0579fbe06.slice/crio-e4640bedac5ac9791617e429d481d2d4dc144b3e24778c76f16df21c7846da1c WatchSource:0}: Error finding container e4640bedac5ac9791617e429d481d2d4dc144b3e24778c76f16df21c7846da1c: Status 404 returned error can't find the container with id e4640bedac5ac9791617e429d481d2d4dc144b3e24778c76f16df21c7846da1c
Jul 19 15:15:00.502382 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.502363    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:15:00.947323 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:00.947231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc" event={"ID":"84bfcad4-da42-4127-992d-d2f0579fbe06","Type":"ContainerStarted","Data":"e4640bedac5ac9791617e429d481d2d4dc144b3e24778c76f16df21c7846da1c"}
Jul 19 15:15:05.971910 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:05.971874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc" event={"ID":"84bfcad4-da42-4127-992d-d2f0579fbe06","Type":"ContainerStarted","Data":"df3bd20fd8f0b15e63259090aff34d2524b97b119a07b0b2163707130ce8aa68"}
Jul 19 15:15:05.988292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:05.988247    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc" podStartSLOduration=1.383161729 podStartE2EDuration="5.988233484s" totalImagesPullingTime="4.605071755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:15:05.98708079 +0000 UTC m=+1687.012089868" watchObservedRunningTime="2026-07-19 15:15:05.988233484 +0000 UTC m=+1687.013242559"
Jul 19 15:15:06.978122 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:06.978087    2577 generic.go:356] "Generic (PLEG): container finished" podID="84bfcad4-da42-4127-992d-d2f0579fbe06" containerID="df3bd20fd8f0b15e63259090aff34d2524b97b119a07b0b2163707130ce8aa68" exitCode=0
Jul 19 15:15:06.978478 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:06.978168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc" event={"ID":"84bfcad4-da42-4127-992d-d2f0579fbe06","Type":"ContainerDied","Data":"df3bd20fd8f0b15e63259090aff34d2524b97b119a07b0b2163707130ce8aa68"}
Jul 19 15:15:08.119055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:08.119031    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"
Jul 19 15:15:08.440059 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:08.439982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:08.990472 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:08.990441    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc"
Jul 19 15:15:08.990472 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:08.990452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741235-x9dhc" event={"ID":"84bfcad4-da42-4127-992d-d2f0579fbe06","Type":"ContainerDied","Data":"e4640bedac5ac9791617e429d481d2d4dc144b3e24778c76f16df21c7846da1c"}
Jul 19 15:15:08.990702 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:08.990514    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4640bedac5ac9791617e429d481d2d4dc144b3e24778c76f16df21c7846da1c"
Jul 19 15:15:09.183020 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:09.182980    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"]
Jul 19 15:15:09.203601 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:09.203574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:24.203884 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.203799    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7" podUID="7b45e4c9-c988-40c5-a304-36c4e22f515b" containerName="main" containerID="cri-o://b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa" gracePeriod=45
Jul 19 15:15:24.462770 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.462712    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:15:24.554426 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554401    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-dshm\") pod \"7b45e4c9-c988-40c5-a304-36c4e22f515b\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") "
Jul 19 15:15:24.554571 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554449    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-home\") pod \"7b45e4c9-c988-40c5-a304-36c4e22f515b\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") "
Jul 19 15:15:24.554571 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-model-cache\") pod \"7b45e4c9-c988-40c5-a304-36c4e22f515b\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") "
Jul 19 15:15:24.554571 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554547    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-tmp-dir\") pod \"7b45e4c9-c988-40c5-a304-36c4e22f515b\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") "
Jul 19 15:15:24.554693 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b45e4c9-c988-40c5-a304-36c4e22f515b-tls-certs\") pod \"7b45e4c9-c988-40c5-a304-36c4e22f515b\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") "
Jul 19 15:15:24.554693 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxvfl\" (UniqueName: \"kubernetes.io/projected/7b45e4c9-c988-40c5-a304-36c4e22f515b-kube-api-access-rxvfl\") pod \"7b45e4c9-c988-40c5-a304-36c4e22f515b\" (UID: \"7b45e4c9-c988-40c5-a304-36c4e22f515b\") "
Jul 19 15:15:24.554801 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554770    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-home" (OuterVolumeSpecName: "home") pod "7b45e4c9-c988-40c5-a304-36c4e22f515b" (UID: "7b45e4c9-c988-40c5-a304-36c4e22f515b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:24.554861 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b45e4c9-c988-40c5-a304-36c4e22f515b" (UID: "7b45e4c9-c988-40c5-a304-36c4e22f515b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:24.554861 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554815    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b45e4c9-c988-40c5-a304-36c4e22f515b" (UID: "7b45e4c9-c988-40c5-a304-36c4e22f515b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:24.554967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554923    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-home\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:15:24.554967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554939    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-model-cache\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:15:24.554967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.554949    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-tmp-dir\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:15:24.556589 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.556568    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-dshm" (OuterVolumeSpecName: "dshm") pod "7b45e4c9-c988-40c5-a304-36c4e22f515b" (UID: "7b45e4c9-c988-40c5-a304-36c4e22f515b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:24.556669 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.556652    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b45e4c9-c988-40c5-a304-36c4e22f515b-kube-api-access-rxvfl" (OuterVolumeSpecName: "kube-api-access-rxvfl") pod "7b45e4c9-c988-40c5-a304-36c4e22f515b" (UID: "7b45e4c9-c988-40c5-a304-36c4e22f515b"). InnerVolumeSpecName "kube-api-access-rxvfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:15:24.556785 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.556764    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b45e4c9-c988-40c5-a304-36c4e22f515b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b45e4c9-c988-40c5-a304-36c4e22f515b" (UID: "7b45e4c9-c988-40c5-a304-36c4e22f515b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:15:24.655532 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.655486    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b45e4c9-c988-40c5-a304-36c4e22f515b-dshm\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:15:24.655532 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.655530    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b45e4c9-c988-40c5-a304-36c4e22f515b-tls-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:15:24.655670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:24.655542    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxvfl\" (UniqueName: \"kubernetes.io/projected/7b45e4c9-c988-40c5-a304-36c4e22f515b-kube-api-access-rxvfl\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:15:25.072078 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.072046    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b45e4c9-c988-40c5-a304-36c4e22f515b" containerID="b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa" exitCode=0
Jul 19 15:15:25.072230 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.072091    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"
Jul 19 15:15:25.072230 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.072137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7" event={"ID":"7b45e4c9-c988-40c5-a304-36c4e22f515b","Type":"ContainerDied","Data":"b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa"}
Jul 19 15:15:25.072230 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.072184    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7" event={"ID":"7b45e4c9-c988-40c5-a304-36c4e22f515b","Type":"ContainerDied","Data":"deef0b160e9965a22c3c1a538b0cda335666b8ad71885334e4c3a78adb693732"}
Jul 19 15:15:25.072230 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.072206    2577 scope.go:117] "RemoveContainer" containerID="b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa"
Jul 19 15:15:25.081230 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.081212    2577 scope.go:117] "RemoveContainer" containerID="b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa"
Jul 19 15:15:25.081506 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:15:25.081475    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa\": container with ID starting with b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa not found: ID does not exist" containerID="b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa"
Jul 19 15:15:25.081560 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.081523    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa"} err="failed to get container status \"b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa\": rpc error: code = NotFound desc = could not find container \"b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa\": container with ID starting with b1aeeafa04dafcb80ae5338f00f3cb83bb7360303012e2fe24f4c5f851ec19fa not found: ID does not exist"
Jul 19 15:15:25.092648 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.092625    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"]
Jul 19 15:15:25.094690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.094671    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-79056b85-kserve-64675496c9-5rgv7"]
Jul 19 15:15:25.539348 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:15:25.539318    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b45e4c9-c988-40c5-a304-36c4e22f515b" path="/var/lib/kubelet/pods/7b45e4c9-c988-40c5-a304-36c4e22f515b/volumes"
Jul 19 15:16:09.613509 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:16:09.613471    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:16.038797 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:16.038757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:23.435143 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:23.435059    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:31.701174 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:31.701138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:36.856365 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:36.856313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:39.984078 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:39.984026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:40.087730 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:40.087689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:40.516433 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:40.516395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:46.354942 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.354894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:46.420937 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.420908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:46.601258 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.601233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:46.905116 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905075    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"]
Jul 19 15:18:46.905593 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905571    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84bfcad4-da42-4127-992d-d2f0579fbe06" containerName="cleanup"
Jul 19 15:18:46.905593 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905594    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="84bfcad4-da42-4127-992d-d2f0579fbe06" containerName="cleanup"
Jul 19 15:18:46.905785 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905642    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b45e4c9-c988-40c5-a304-36c4e22f515b" containerName="main"
Jul 19 15:18:46.905785 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905650    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b45e4c9-c988-40c5-a304-36c4e22f515b" containerName="main"
Jul 19 15:18:46.905785 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905735    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="84bfcad4-da42-4127-992d-d2f0579fbe06" containerName="cleanup"
Jul 19 15:18:46.905785 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.905750    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b45e4c9-c988-40c5-a304-36c4e22f515b" containerName="main"
Jul 19 15:18:46.908859 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.908828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.911357 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.911335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-db501197-openshift-default-dockercfg-l8kcr\""
Jul 19 15:18:46.933825 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.933792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"]
Jul 19 15:18:46.940216 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.940175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:46.957059 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-socket\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957151 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-envoy\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957151 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-data\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957151 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5218a-8342-4735-8294-ec4ce45b7127-istiod-ca-cert\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957258 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-certs\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957258 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxngs\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-kube-api-access-bxngs\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957330 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-credential-socket\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957330 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-token\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:46.957394 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:46.957351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-podinfo\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058199 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-socket\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058199 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-envoy\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-data\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5218a-8342-4735-8294-ec4ce45b7127-istiod-ca-cert\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-certs\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxngs\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-kube-api-access-bxngs\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-credential-socket\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058736 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-token\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058736 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-podinfo\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058736 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-data\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058897 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-certs\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.058897 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-credential-socket\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.059009 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.058909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-socket\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.059316 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.059296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5218a-8342-4735-8294-ec4ce45b7127-istiod-ca-cert\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.060656 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.060637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-envoy\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.060879 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.060860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-podinfo\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.066308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.066289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-token\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.066590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.066564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxngs\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-kube-api-access-bxngs\") pod \"e2e-ait-db501197-openshift-default-f7fb6945c-h5g48\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") " pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.222352 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.222278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:47.366359 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.366335    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"]
Jul 19 15:18:47.369434 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:18:47.369396    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b5218a_8342_4735_8294_ec4ce45b7127.slice/crio-ffbebb1787b8aa4f39cb371363ec2528a4fbd616a6dcc3a35060ebb4e401658a WatchSource:0}: Error finding container ffbebb1787b8aa4f39cb371363ec2528a4fbd616a6dcc3a35060ebb4e401658a: Status 404 returned error can't find the container with id ffbebb1787b8aa4f39cb371363ec2528a4fbd616a6dcc3a35060ebb4e401658a
Jul 19 15:18:47.372013 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.371985    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:18:47.372094 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.372051    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:18:47.372094 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.372078    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:18:47.507079 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:47.507049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:48.038637 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:48.038603    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" event={"ID":"b9b5218a-8342-4735-8294-ec4ce45b7127","Type":"ContainerStarted","Data":"6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"}
Jul 19 15:18:48.038822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:48.038662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" event={"ID":"b9b5218a-8342-4735-8294-ec4ce45b7127","Type":"ContainerStarted","Data":"ffbebb1787b8aa4f39cb371363ec2528a4fbd616a6dcc3a35060ebb4e401658a"}
Jul 19 15:18:48.061464 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:48.061403    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podStartSLOduration=2.061382167 podStartE2EDuration="2.061382167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:18:48.058935017 +0000 UTC m=+1909.083944093" watchObservedRunningTime="2026-07-19 15:18:48.061382167 +0000 UTC m=+1909.086391243"
Jul 19 15:18:48.223333 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:48.223302    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:48.227986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:48.227965    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:49.043579 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.043552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:49.044583 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.044564    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:18:49.410586 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.410504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:49.513050 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.513020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:49.578821 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.578790    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"]
Jul 19 15:18:49.582445 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.582424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.584663 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.584645    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-db501197-serving-cert\""
Jul 19 15:18:49.593713 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.593692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"]
Jul 19 15:18:49.607598 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.607571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:49.680832 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.680752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llq5t\" (UniqueName: \"kubernetes.io/projected/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-kube-api-access-llq5t\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.681014 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.680876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-openshift-service-ca\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.681014 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.680935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.781916 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.781877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-openshift-service-ca\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.782105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.781922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.782179 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:18:49.782110    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-db501197-serving-cert: secret "maas-api-e2e-ait-db501197-serving-cert" not found
Jul 19 15:18:49.782179 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.782117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llq5t\" (UniqueName: \"kubernetes.io/projected/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-kube-api-access-llq5t\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.782179 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:18:49.782176    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls podName:e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872 nodeName:}" failed. No retries permitted until 2026-07-19 15:18:50.282155524 +0000 UTC m=+1911.307164593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls") pod "maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" (UID: "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872") : secret "maas-api-e2e-ait-db501197-serving-cert" not found
Jul 19 15:18:49.782661 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.782636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-openshift-service-ca\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.790733 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.790711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llq5t\" (UniqueName: \"kubernetes.io/projected/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-kube-api-access-llq5t\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:49.859410 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:49.859378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:50.063441 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:50.063409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:50.164632 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:50.164602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:50.285985 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:50.285950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:50.288390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:50.288361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls\") pod \"maas-api-e2e-ait-db501197-ccb8d89d8-98h4v\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:50.493905 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:50.493876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:50.629730 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:50.629699    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"]
Jul 19 15:18:50.631573 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:18:50.631544    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88e2f6e_bfa8_49ce_8f73_d4bd1ea51872.slice/crio-a09fe26e95263ebff72872aa4affc6bbd5eff3bcf85d2fac089afde3da5318f4 WatchSource:0}: Error finding container a09fe26e95263ebff72872aa4affc6bbd5eff3bcf85d2fac089afde3da5318f4: Status 404 returned error can't find the container with id a09fe26e95263ebff72872aa4affc6bbd5eff3bcf85d2fac089afde3da5318f4
Jul 19 15:18:51.054247 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:51.054203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" event={"ID":"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872","Type":"ContainerStarted","Data":"a09fe26e95263ebff72872aa4affc6bbd5eff3bcf85d2fac089afde3da5318f4"}
Jul 19 15:18:52.059983 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:52.059948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" event={"ID":"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872","Type":"ContainerStarted","Data":"f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c"}
Jul 19 15:18:52.060355 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:52.060016    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:18:52.076983 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:52.076935    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" podStartSLOduration=2.6110901589999997 podStartE2EDuration="3.076918703s" totalImagesPullingTime="465.828544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:18:52.075385798 +0000 UTC m=+1913.100394873" watchObservedRunningTime="2026-07-19 15:18:52.076918703 +0000 UTC m=+1913.101927779"
Jul 19 15:18:58.069947 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:18:58.069911    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:19:04.045353 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.045318    2577 patch_prober.go:28] interesting pod/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:19:04.045715 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.045393    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:19:04.118471 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.118447    2577 generic.go:356] "Generic (PLEG): container finished" podID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerID="6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001" exitCode=0
Jul 19 15:19:04.118588 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.118488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" event={"ID":"b9b5218a-8342-4735-8294-ec4ce45b7127","Type":"ContainerDied","Data":"6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"}
Jul 19 15:19:04.118888 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.118871    2577 scope.go:117] "RemoveContainer" containerID="6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"
Jul 19 15:19:04.119838 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.119810    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:04.119896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.119880    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:04.119972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:04.119908    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:05.132627 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:05.132588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" event={"ID":"b9b5218a-8342-4735-8294-ec4ce45b7127","Type":"ContainerStarted","Data":"d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d"}
Jul 19 15:19:05.223369 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:05.223327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:19:05.223583 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:05.223392    2577 patch_prober.go:28] interesting pod/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:19:05.223583 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:05.223441    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:19:06.223796 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:06.223752    2577 patch_prober.go:28] interesting pod/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:19:06.224295 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:06.223821    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:19:07.224104 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:07.223648    2577 patch_prober.go:28] interesting pod/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:19:07.224104 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:07.223711    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:19:08.223033 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:08.222993    2577 patch_prober.go:28] interesting pod/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:19:08.223236 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:08.223056    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:19:09.227270 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:09.227243    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:19:09.227708 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:09.227590    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:19:09.228139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:09.228124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:19:15.053603 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.053567    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"]
Jul 19 15:19:15.055912 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.055887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:15.058452 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.058428    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:19:15.066156 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.066132    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"]
Jul 19 15:19:15.106832 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.106802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ac10-15e5-4e10-9ec2-776bff549f41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-db501197-7l7h2\" (UID: \"ef70ac10-15e5-4e10-9ec2-776bff549f41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:15.208252 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.208213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ac10-15e5-4e10-9ec2-776bff549f41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-db501197-7l7h2\" (UID: \"ef70ac10-15e5-4e10-9ec2-776bff549f41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:15.208888 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.208861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ac10-15e5-4e10-9ec2-776bff549f41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-db501197-7l7h2\" (UID: \"ef70ac10-15e5-4e10-9ec2-776bff549f41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:15.371062 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.370972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:15.503758 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:15.503724    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"]
Jul 19 15:19:15.504483 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:19:15.504458    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef70ac10_15e5_4e10_9ec2_776bff549f41.slice/crio-0b5d37702434e01b783bb5ba0cb32697c7f7896d31da8424c702d1a11f766f70 WatchSource:0}: Error finding container 0b5d37702434e01b783bb5ba0cb32697c7f7896d31da8424c702d1a11f766f70: Status 404 returned error can't find the container with id 0b5d37702434e01b783bb5ba0cb32697c7f7896d31da8424c702d1a11f766f70
Jul 19 15:19:16.189631 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:16.189593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2" event={"ID":"ef70ac10-15e5-4e10-9ec2-776bff549f41","Type":"ContainerStarted","Data":"282bd042f623e9a811e78054e8569ad369442a0b80c3e40a0809c255514d8e6d"}
Jul 19 15:19:16.189966 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:16.189655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2" event={"ID":"ef70ac10-15e5-4e10-9ec2-776bff549f41","Type":"ContainerStarted","Data":"0b5d37702434e01b783bb5ba0cb32697c7f7896d31da8424c702d1a11f766f70"}
Jul 19 15:19:16.204933 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:16.204886    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2" podStartSLOduration=1.204871073 podStartE2EDuration="1.204871073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:19:16.203403449 +0000 UTC m=+1937.228412525" watchObservedRunningTime="2026-07-19 15:19:16.204871073 +0000 UTC m=+1937.229880145"
Jul 19 15:19:17.195979 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:17.195949    2577 generic.go:356] "Generic (PLEG): container finished" podID="ef70ac10-15e5-4e10-9ec2-776bff549f41" containerID="282bd042f623e9a811e78054e8569ad369442a0b80c3e40a0809c255514d8e6d" exitCode=0
Jul 19 15:19:17.196383 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:17.195993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2" event={"ID":"ef70ac10-15e5-4e10-9ec2-776bff549f41","Type":"ContainerDied","Data":"282bd042f623e9a811e78054e8569ad369442a0b80c3e40a0809c255514d8e6d"}
Jul 19 15:19:18.334305 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:18.334285    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:18.430511 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:18.430464    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ac10-15e5-4e10-9ec2-776bff549f41-maas-api-service-ca\") pod \"ef70ac10-15e5-4e10-9ec2-776bff549f41\" (UID: \"ef70ac10-15e5-4e10-9ec2-776bff549f41\") "
Jul 19 15:19:18.430808 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:18.430788    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef70ac10-15e5-4e10-9ec2-776bff549f41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef70ac10-15e5-4e10-9ec2-776bff549f41" (UID: "ef70ac10-15e5-4e10-9ec2-776bff549f41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:19:18.531310 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:18.531281    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ac10-15e5-4e10-9ec2-776bff549f41-maas-api-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:19.205899 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:19.205874    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"
Jul 19 15:19:19.206070 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:19.205869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2" event={"ID":"ef70ac10-15e5-4e10-9ec2-776bff549f41","Type":"ContainerDied","Data":"0b5d37702434e01b783bb5ba0cb32697c7f7896d31da8424c702d1a11f766f70"}
Jul 19 15:19:19.206070 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:19.205962    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b5d37702434e01b783bb5ba0cb32697c7f7896d31da8424c702d1a11f766f70"
Jul 19 15:19:25.417321 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.417284    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"]
Jul 19 15:19:25.417763 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.417470    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" podUID="e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" containerName="maas-api" containerID="cri-o://f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c" gracePeriod=30
Jul 19 15:19:25.452388 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.452360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:25.660480 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.660460    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:19:25.686024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.685965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-openshift-service-ca\") pod \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") "
Jul 19 15:19:25.686141 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.686038    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llq5t\" (UniqueName: \"kubernetes.io/projected/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-kube-api-access-llq5t\") pod \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") "
Jul 19 15:19:25.686141 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.686079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls\") pod \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\" (UID: \"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872\") "
Jul 19 15:19:25.686299 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.686277    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" (UID: "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:19:25.686363 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.686351    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:25.688120 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.688102    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" (UID: "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:19:25.688263 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.688243    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-kube-api-access-llq5t" (OuterVolumeSpecName: "kube-api-access-llq5t") pod "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" (UID: "e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872"). InnerVolumeSpecName "kube-api-access-llq5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:19:25.786749 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.786725    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llq5t\" (UniqueName: \"kubernetes.io/projected/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-kube-api-access-llq5t\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:25.786749 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:25.786747    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:26.242737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.242707    2577 generic.go:356] "Generic (PLEG): container finished" podID="e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" containerID="f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c" exitCode=0
Jul 19 15:19:26.242917 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.242747    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"
Jul 19 15:19:26.242917 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.242798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" event={"ID":"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872","Type":"ContainerDied","Data":"f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c"}
Jul 19 15:19:26.242917 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.242844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v" event={"ID":"e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872","Type":"ContainerDied","Data":"a09fe26e95263ebff72872aa4affc6bbd5eff3bcf85d2fac089afde3da5318f4"}
Jul 19 15:19:26.242917 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.242867    2577 scope.go:117] "RemoveContainer" containerID="f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c"
Jul 19 15:19:26.251776 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.251758    2577 scope.go:117] "RemoveContainer" containerID="f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c"
Jul 19 15:19:26.252017 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:19:26.252000    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c\": container with ID starting with f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c not found: ID does not exist" containerID="f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c"
Jul 19 15:19:26.252072 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.252027    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c"} err="failed to get container status \"f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c\": rpc error: code = NotFound desc = could not find container \"f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c\": container with ID starting with f00730ef344dc975b6bb6db2bd5b6b63ace50dea8bc55e394fbe38ce044d912c not found: ID does not exist"
Jul 19 15:19:26.263277 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.263257    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"]
Jul 19 15:19:26.264994 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:26.264975    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db501197-ccb8d89d8-98h4v"]
Jul 19 15:19:27.539578 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:27.539546    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" path="/var/lib/kubelet/pods/e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872/volumes"
Jul 19 15:19:30.118048 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.118020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"]
Jul 19 15:19:30.121065 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.121040    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-db501197-7l7h2"]
Jul 19 15:19:30.123055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.123029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:30.267557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.267525    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"]
Jul 19 15:19:30.267853 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.267791    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy" containerID="cri-o://d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d" gracePeriod=30
Jul 19 15:19:30.286838 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.286814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:30.529580 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.529532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:30.789479 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.789399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:30.882773 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:30.882741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:31.014707 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.014672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:31.307846 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.307818    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"]
Jul 19 15:19:31.308257 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.308240    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" containerName="maas-api"
Jul 19 15:19:31.308303 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.308259    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" containerName="maas-api"
Jul 19 15:19:31.308303 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.308274    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef70ac10-15e5-4e10-9ec2-776bff549f41" containerName="revoke-keys"
Jul 19 15:19:31.308303 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.308280    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef70ac10-15e5-4e10-9ec2-776bff549f41" containerName="revoke-keys"
Jul 19 15:19:31.308397 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.308355    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef70ac10-15e5-4e10-9ec2-776bff549f41" containerName="revoke-keys"
Jul 19 15:19:31.308397 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.308364    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e88e2f6e-bfa8-49ce-8f73-d4bd1ea51872" containerName="maas-api"
Jul 19 15:19:31.310385 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.310366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.312659 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.312640    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6d22b79c-gw-openshift-default-dockercfg-mshst\""
Jul 19 15:19:31.322772 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.322747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"]
Jul 19 15:19:31.331432 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.331402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:31.333286 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istiod-ca-cert\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333396 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-socket\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333396 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-data\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333396 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66lp9\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-kube-api-access-66lp9\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333547 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-envoy\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333624 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-credential-socket\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333672 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-token\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333746 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-certs\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.333746 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.333692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-podinfo\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.434478 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-credential-socket\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.434478 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-token\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.434755 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-certs\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.434755 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-podinfo\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.434755 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istiod-ca-cert\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.434945 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-credential-socket\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435058 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.434969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-certs\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435164 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-socket\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435300 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-socket\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-data\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66lp9\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-kube-api-access-66lp9\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-envoy\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435814    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istiod-ca-cert\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.435880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.435863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-data\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.443469 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.441767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-podinfo\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.445892 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.445849    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-envoy\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.445982 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.445939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-token\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.450868 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.450851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66lp9\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-kube-api-access-66lp9\") pod \"e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") " pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.539267 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.539238    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef70ac10-15e5-4e10-9ec2-776bff549f41" path="/var/lib/kubelet/pods/ef70ac10-15e5-4e10-9ec2-776bff549f41/volumes"
Jul 19 15:19:31.622121 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.622051    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:31.766192 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.766087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"]
Jul 19 15:19:31.768400 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:19:31.768372    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8325ffdb_ebe2_4bae_9bfb_8900d32705b4.slice/crio-884f1f4a939394a17a893bca7fc0d969baec6b411e77a4879031b41f20e63b41 WatchSource:0}: Error finding container 884f1f4a939394a17a893bca7fc0d969baec6b411e77a4879031b41f20e63b41: Status 404 returned error can't find the container with id 884f1f4a939394a17a893bca7fc0d969baec6b411e77a4879031b41f20e63b41
Jul 19 15:19:31.771025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.770978    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:31.771106 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.771060    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:31.771106 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.771088    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:31.805685 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:31.805643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:32.277195 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:32.277163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" event={"ID":"8325ffdb-ebe2-4bae-9bfb-8900d32705b4","Type":"ContainerStarted","Data":"e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"}
Jul 19 15:19:32.277360 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:32.277219    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" event={"ID":"8325ffdb-ebe2-4bae-9bfb-8900d32705b4","Type":"ContainerStarted","Data":"884f1f4a939394a17a893bca7fc0d969baec6b411e77a4879031b41f20e63b41"}
Jul 19 15:19:32.296704 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:32.296661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" podStartSLOduration=1.296647014 podStartE2EDuration="1.296647014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:19:32.295232365 +0000 UTC m=+1953.320241441" watchObservedRunningTime="2026-07-19 15:19:32.296647014 +0000 UTC m=+1953.321656088"
Jul 19 15:19:32.622563 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:32.622473    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:32.627074 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:32.627048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:33.282452 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.282422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:33.283287 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.283267    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:33.747554 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.747517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:33.818247 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.818216    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"]
Jul 19 15:19:33.821273 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.821250    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.823678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.823651    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6d22b79c-serving-cert\""
Jul 19 15:19:33.831762 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.831742    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"]
Jul 19 15:19:33.848279 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.848253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:33.859383 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.859356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvz4c\" (UniqueName: \"kubernetes.io/projected/d23b3a30-83e7-43f9-8397-f10b8ceb8648-kube-api-access-lvz4c\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.859536 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.859405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23b3a30-83e7-43f9-8397-f10b8ceb8648-openshift-service-ca\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.859536 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.859437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23b3a30-83e7-43f9-8397-f10b8ceb8648-maas-api-tls\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.947973 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.947932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:33.960646 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.960623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvz4c\" (UniqueName: \"kubernetes.io/projected/d23b3a30-83e7-43f9-8397-f10b8ceb8648-kube-api-access-lvz4c\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.960788 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.960653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23b3a30-83e7-43f9-8397-f10b8ceb8648-openshift-service-ca\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.960788 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.960671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23b3a30-83e7-43f9-8397-f10b8ceb8648-maas-api-tls\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.961246 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.961224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23b3a30-83e7-43f9-8397-f10b8ceb8648-openshift-service-ca\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.963115 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.963094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23b3a30-83e7-43f9-8397-f10b8ceb8648-maas-api-tls\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:33.969396 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:33.969365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvz4c\" (UniqueName: \"kubernetes.io/projected/d23b3a30-83e7-43f9-8397-f10b8ceb8648-kube-api-access-lvz4c\") pod \"maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:34.135129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:34.135047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:34.413277 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:34.413201    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:34.470976 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:34.470946    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"]
Jul 19 15:19:34.473154 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:34.473118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:34.473752 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:19:34.473726    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23b3a30_83e7_43f9_8397_f10b8ceb8648.slice/crio-11c0034dd7e79db5d4f765cf66908ec6cd0e64387bfdb8b8d5c863bb1d4331ff WatchSource:0}: Error finding container 11c0034dd7e79db5d4f765cf66908ec6cd0e64387bfdb8b8d5c863bb1d4331ff: Status 404 returned error can't find the container with id 11c0034dd7e79db5d4f765cf66908ec6cd0e64387bfdb8b8d5c863bb1d4331ff
Jul 19 15:19:35.294005 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.293974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz" event={"ID":"d23b3a30-83e7-43f9-8397-f10b8ceb8648","Type":"ContainerStarted","Data":"1a635345a3970c731f4baca75de84d84215ab0fe4d14724889be879a02344b37"}
Jul 19 15:19:35.294417 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.294034    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz" event={"ID":"d23b3a30-83e7-43f9-8397-f10b8ceb8648","Type":"ContainerStarted","Data":"11c0034dd7e79db5d4f765cf66908ec6cd0e64387bfdb8b8d5c863bb1d4331ff"}
Jul 19 15:19:35.294417 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.294201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:35.310337 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.310300    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz" podStartSLOduration=1.667448674 podStartE2EDuration="2.310285963s" totalImagesPullingTime="642.837289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:19:35.308745342 +0000 UTC m=+1956.333754426" watchObservedRunningTime="2026-07-19 15:19:35.310285963 +0000 UTC m=+1956.335295037"
Jul 19 15:19:35.523728 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.523703    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:19:35.576829 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.576804    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-credential-socket\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.576963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.576859    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-envoy\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.576963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.576878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxngs\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-kube-api-access-bxngs\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.576963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.576903    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-data\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.576963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.576939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-socket\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.577181 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.576965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-token\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.577181 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5218a-8342-4735-8294-ec4ce45b7127-istiod-ca-cert\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.577181 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577052    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-podinfo\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.577181 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:19:35.577181 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577094    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-certs\") pod \"b9b5218a-8342-4735-8294-ec4ce45b7127\" (UID: \"b9b5218a-8342-4735-8294-ec4ce45b7127\") "
Jul 19 15:19:35.577942 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577666    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:19:35.577942 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577931    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.578133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577954    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.578133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.577967    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b5218a-8342-4735-8294-ec4ce45b7127-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:19:35.578133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.578057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:19:35.578303 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.578230    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:19:35.579880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.579844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:19:35.581539 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.580401    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-kube-api-access-bxngs" (OuterVolumeSpecName: "kube-api-access-bxngs") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "kube-api-access-bxngs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:19:35.581539 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.580868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:19:35.585949 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.585928    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9b5218a-8342-4735-8294-ec4ce45b7127" (UID: "b9b5218a-8342-4735-8294-ec4ce45b7127"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:19:35.678462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678406    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.678462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678427    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5218a-8342-4735-8294-ec4ce45b7127-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.678462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678435    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.678462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678442    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.678462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678451    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.678462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678460    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxngs\" (UniqueName: \"kubernetes.io/projected/b9b5218a-8342-4735-8294-ec4ce45b7127-kube-api-access-bxngs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:35.678726 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:35.678468    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b5218a-8342-4735-8294-ec4ce45b7127-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:19:36.300091 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.300058    2577 generic.go:356] "Generic (PLEG): container finished" podID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerID="d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d" exitCode=0
Jul 19 15:19:36.300476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.300119    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"
Jul 19 15:19:36.300476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.300138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" event={"ID":"b9b5218a-8342-4735-8294-ec4ce45b7127","Type":"ContainerDied","Data":"d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d"}
Jul 19 15:19:36.300476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.300229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48" event={"ID":"b9b5218a-8342-4735-8294-ec4ce45b7127","Type":"ContainerDied","Data":"ffbebb1787b8aa4f39cb371363ec2528a4fbd616a6dcc3a35060ebb4e401658a"}
Jul 19 15:19:36.300476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.300251    2577 scope.go:117] "RemoveContainer" containerID="d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d"
Jul 19 15:19:36.309246 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.309229    2577 scope.go:117] "RemoveContainer" containerID="6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"
Jul 19 15:19:36.316903 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.316890    2577 scope.go:117] "RemoveContainer" containerID="d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d"
Jul 19 15:19:36.317126 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:19:36.317110    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d\": container with ID starting with d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d not found: ID does not exist" containerID="d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d"
Jul 19 15:19:36.317170 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.317135    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d"} err="failed to get container status \"d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d\": rpc error: code = NotFound desc = could not find container \"d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d\": container with ID starting with d0d9bf1d26259533eabcf0a113b74541eac6bdf7513e803403476761c6d97e7d not found: ID does not exist"
Jul 19 15:19:36.317170 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.317151    2577 scope.go:117] "RemoveContainer" containerID="6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"
Jul 19 15:19:36.317353 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:19:36.317340    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001\": container with ID starting with 6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001 not found: ID does not exist" containerID="6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"
Jul 19 15:19:36.317395 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.317356    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001"} err="failed to get container status \"6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001\": rpc error: code = NotFound desc = could not find container \"6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001\": container with ID starting with 6111c1df56adc51c6929cdc840ab8e4484590cfadd9a7e305e62cfd4b4c4c001 not found: ID does not exist"
Jul 19 15:19:36.337156 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.337137    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"]
Jul 19 15:19:36.339961 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:36.339938    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-db501197-openshift-default-f7fb6945c-h5g48"]
Jul 19 15:19:37.540342 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:37.540310    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" path="/var/lib/kubelet/pods/b9b5218a-8342-4735-8294-ec4ce45b7127/volumes"
Jul 19 15:19:41.307201 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:41.307167    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:19:47.373634 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:47.373600    2577 generic.go:356] "Generic (PLEG): container finished" podID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerID="e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a" exitCode=0
Jul 19 15:19:47.374171 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:47.373652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" event={"ID":"8325ffdb-ebe2-4bae-9bfb-8900d32705b4","Type":"ContainerDied","Data":"e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"}
Jul 19 15:19:47.374171 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:47.373990    2577 scope.go:117] "RemoveContainer" containerID="e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"
Jul 19 15:19:47.374919 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:47.374893    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:47.375015 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:47.374970    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:47.375068 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:47.375021    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:19:48.282604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:48.282571    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:48.380895 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:48.380861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" event={"ID":"8325ffdb-ebe2-4bae-9bfb-8900d32705b4","Type":"ContainerStarted","Data":"e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77"}
Jul 19 15:19:48.624139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:48.624036    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:48.624430 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:48.624208    2577 patch_prober.go:28] interesting pod/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 19 15:19:48.624430 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:48.624253    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 19 15:19:49.622637 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:49.622602    2577 patch_prober.go:28] interesting pod/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 19 15:19:49.623155 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:49.622666    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 19 15:19:50.622846 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:50.622807    2577 patch_prober.go:28] interesting pod/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 19 15:19:50.623477 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:50.623447    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 19 15:19:51.622983 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:51.622868    2577 patch_prober.go:28] interesting pod/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 19 15:19:51.622983 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:51.622935    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 19 15:19:52.626596 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:52.626569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:52.627050 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:52.626822    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:52.627488 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:52.627473    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:19:58.810063 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.810029    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"]
Jul 19 15:19:58.810443 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.810379    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy"
Jul 19 15:19:58.810443 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.810389    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy"
Jul 19 15:19:58.810557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.810459    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy"
Jul 19 15:19:58.810557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.810467    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy"
Jul 19 15:19:58.813685 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.813667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:19:58.816001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.815975    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:19:58.820853 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.820825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"]
Jul 19 15:19:58.987294 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:58.987256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5b4ad1c-61dd-49b1-a266-bc0b373f9714-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp\" (UID: \"b5b4ad1c-61dd-49b1-a266-bc0b373f9714\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:19:59.088486 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:59.088404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5b4ad1c-61dd-49b1-a266-bc0b373f9714-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp\" (UID: \"b5b4ad1c-61dd-49b1-a266-bc0b373f9714\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:19:59.089036 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:59.089017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5b4ad1c-61dd-49b1-a266-bc0b373f9714-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp\" (UID: \"b5b4ad1c-61dd-49b1-a266-bc0b373f9714\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:19:59.125967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:59.125941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:19:59.256247 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:59.256211    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"]
Jul 19 15:19:59.259555 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:19:59.259524    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b4ad1c_61dd_49b1_a266_bc0b373f9714.slice/crio-51a0be4c359499734eb6a22536022b4403801e9d58cb3745269577f67831e029 WatchSource:0}: Error finding container 51a0be4c359499734eb6a22536022b4403801e9d58cb3745269577f67831e029: Status 404 returned error can't find the container with id 51a0be4c359499734eb6a22536022b4403801e9d58cb3745269577f67831e029
Jul 19 15:19:59.436877 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:19:59.436839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp" event={"ID":"b5b4ad1c-61dd-49b1-a266-bc0b373f9714","Type":"ContainerStarted","Data":"51a0be4c359499734eb6a22536022b4403801e9d58cb3745269577f67831e029"}
Jul 19 15:20:00.443129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:00.443049    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5b4ad1c-61dd-49b1-a266-bc0b373f9714" containerID="c179d41efa41bf56eb6313a8627aab1e3cdd1dc91f58cee772ab25277d8dee27" exitCode=0
Jul 19 15:20:00.443466 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:00.443132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp" event={"ID":"b5b4ad1c-61dd-49b1-a266-bc0b373f9714","Type":"ContainerDied","Data":"c179d41efa41bf56eb6313a8627aab1e3cdd1dc91f58cee772ab25277d8dee27"}
Jul 19 15:20:01.583345 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:01.583322    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:20:01.714926 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:01.714827    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5b4ad1c-61dd-49b1-a266-bc0b373f9714-maas-api-service-ca\") pod \"b5b4ad1c-61dd-49b1-a266-bc0b373f9714\" (UID: \"b5b4ad1c-61dd-49b1-a266-bc0b373f9714\") "
Jul 19 15:20:01.715218 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:01.715197    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5b4ad1c-61dd-49b1-a266-bc0b373f9714-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b5b4ad1c-61dd-49b1-a266-bc0b373f9714" (UID: "b5b4ad1c-61dd-49b1-a266-bc0b373f9714"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:20:01.816541 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:01.816485    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5b4ad1c-61dd-49b1-a266-bc0b373f9714-maas-api-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:02.454641 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:02.454607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp" event={"ID":"b5b4ad1c-61dd-49b1-a266-bc0b373f9714","Type":"ContainerDied","Data":"51a0be4c359499734eb6a22536022b4403801e9d58cb3745269577f67831e029"}
Jul 19 15:20:02.454641 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:02.454624    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"
Jul 19 15:20:02.454894 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:02.454653    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51a0be4c359499734eb6a22536022b4403801e9d58cb3745269577f67831e029"
Jul 19 15:20:09.255397 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.255367    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"]
Jul 19 15:20:09.255802 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.255593    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz" podUID="d23b3a30-83e7-43f9-8397-f10b8ceb8648" containerName="maas-api" containerID="cri-o://1a635345a3970c731f4baca75de84d84215ab0fe4d14724889be879a02344b37" gracePeriod=30
Jul 19 15:20:09.274566 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.274537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:09.489574 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.489546    2577 generic.go:356] "Generic (PLEG): container finished" podID="d23b3a30-83e7-43f9-8397-f10b8ceb8648" containerID="1a635345a3970c731f4baca75de84d84215ab0fe4d14724889be879a02344b37" exitCode=0
Jul 19 15:20:09.489705 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.489580    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz" event={"ID":"d23b3a30-83e7-43f9-8397-f10b8ceb8648","Type":"ContainerDied","Data":"1a635345a3970c731f4baca75de84d84215ab0fe4d14724889be879a02344b37"}
Jul 19 15:20:09.506952 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.506898    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:20:09.571215 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.571189    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvz4c\" (UniqueName: \"kubernetes.io/projected/d23b3a30-83e7-43f9-8397-f10b8ceb8648-kube-api-access-lvz4c\") pod \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") "
Jul 19 15:20:09.571346 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.571227    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23b3a30-83e7-43f9-8397-f10b8ceb8648-maas-api-tls\") pod \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") "
Jul 19 15:20:09.573167 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.573137    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23b3a30-83e7-43f9-8397-f10b8ceb8648-kube-api-access-lvz4c" (OuterVolumeSpecName: "kube-api-access-lvz4c") pod "d23b3a30-83e7-43f9-8397-f10b8ceb8648" (UID: "d23b3a30-83e7-43f9-8397-f10b8ceb8648"). InnerVolumeSpecName "kube-api-access-lvz4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:20:09.573261 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.573199    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23b3a30-83e7-43f9-8397-f10b8ceb8648-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d23b3a30-83e7-43f9-8397-f10b8ceb8648" (UID: "d23b3a30-83e7-43f9-8397-f10b8ceb8648"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:20:09.672156 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.672127    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23b3a30-83e7-43f9-8397-f10b8ceb8648-openshift-service-ca\") pod \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\" (UID: \"d23b3a30-83e7-43f9-8397-f10b8ceb8648\") "
Jul 19 15:20:09.672306 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.672294    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvz4c\" (UniqueName: \"kubernetes.io/projected/d23b3a30-83e7-43f9-8397-f10b8ceb8648-kube-api-access-lvz4c\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:09.672345 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.672308    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23b3a30-83e7-43f9-8397-f10b8ceb8648-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:09.672451 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.672430    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23b3a30-83e7-43f9-8397-f10b8ceb8648-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d23b3a30-83e7-43f9-8397-f10b8ceb8648" (UID: "d23b3a30-83e7-43f9-8397-f10b8ceb8648"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:20:09.772825 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:09.772764    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23b3a30-83e7-43f9-8397-f10b8ceb8648-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:10.495505 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:10.495455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz" event={"ID":"d23b3a30-83e7-43f9-8397-f10b8ceb8648","Type":"ContainerDied","Data":"11c0034dd7e79db5d4f765cf66908ec6cd0e64387bfdb8b8d5c863bb1d4331ff"}
Jul 19 15:20:10.495937 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:10.495468    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"
Jul 19 15:20:10.495937 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:10.495546    2577 scope.go:117] "RemoveContainer" containerID="1a635345a3970c731f4baca75de84d84215ab0fe4d14724889be879a02344b37"
Jul 19 15:20:10.515461 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:10.515439    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"]
Jul 19 15:20:10.518763 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:10.518745    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d22b79c-7c77bd6969-5h2jz"]
Jul 19 15:20:11.539333 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:11.539300    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23b3a30-83e7-43f9-8397-f10b8ceb8648" path="/var/lib/kubelet/pods/d23b3a30-83e7-43f9-8397-f10b8ceb8648/volumes"
Jul 19 15:20:13.938585 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:13.938551    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"]
Jul 19 15:20:13.941058 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:13.941034    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d22b79c-8zpkp"]
Jul 19 15:20:13.941199 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:13.941153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:14.087794 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.087759    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"]
Jul 19 15:20:14.088473 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.088435    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy" containerID="cri-o://e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77" gracePeriod=30
Jul 19 15:20:14.101891 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.101868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:14.351365 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.351334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:14.580729 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.580700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:14.877042 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877010    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"]
Jul 19 15:20:14.877427 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877412    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5b4ad1c-61dd-49b1-a266-bc0b373f9714" containerName="revoke-keys"
Jul 19 15:20:14.877474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877429    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5b4ad1c-61dd-49b1-a266-bc0b373f9714" containerName="revoke-keys"
Jul 19 15:20:14.877474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877443    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23b3a30-83e7-43f9-8397-f10b8ceb8648" containerName="maas-api"
Jul 19 15:20:14.877474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877449    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23b3a30-83e7-43f9-8397-f10b8ceb8648" containerName="maas-api"
Jul 19 15:20:14.877474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877473    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy"
Jul 19 15:20:14.877642 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877479    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b5218a-8342-4735-8294-ec4ce45b7127" containerName="istio-proxy"
Jul 19 15:20:14.877642 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877551    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23b3a30-83e7-43f9-8397-f10b8ceb8648" containerName="maas-api"
Jul 19 15:20:14.877642 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.877564    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5b4ad1c-61dd-49b1-a266-bc0b373f9714" containerName="revoke-keys"
Jul 19 15:20:14.881945 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.881930    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.884292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.884274    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b0787d8c-openshift-default-dockercfg-4sfvd\""
Jul 19 15:20:14.890447 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.890421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"]
Jul 19 15:20:14.898236 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.898212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:14.910379 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-istio-token\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910506 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-data\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910506 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487k6\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-kube-api-access-487k6\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910506 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-socket\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910506 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-credential-socket\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1514623-f186-4d20-a6d6-26afd74d907a-istio-podinfo\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1514623-f186-4d20-a6d6-26afd74d907a-istiod-ca-cert\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-certs\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:14.910745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:14.910645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-envoy\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.011548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-socket\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-credential-socket\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1514623-f186-4d20-a6d6-26afd74d907a-istio-podinfo\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1514623-f186-4d20-a6d6-26afd74d907a-istiod-ca-cert\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-certs\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-envoy\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-istio-token\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-data\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487k6\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-kube-api-access-487k6\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-socket\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.011931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-credential-socket\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012552 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.012147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-certs\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012552 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.012374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-data\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.012872 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.012850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1514623-f186-4d20-a6d6-26afd74d907a-istiod-ca-cert\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.014896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.014870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1514623-f186-4d20-a6d6-26afd74d907a-istio-podinfo\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.015115 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.015096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-envoy\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.020648 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.020625    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-istio-token\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.020789 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.020773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-487k6\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-kube-api-access-487k6\") pod \"e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") " pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.194118 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.194037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:15.349965 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.349924    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"]
Jul 19 15:20:15.350147 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:20:15.350126    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1514623_f186_4d20_a6d6_26afd74d907a.slice/crio-08aac748590d55431ee9d292cf41e4c40f15953eb628bc556155cffd081c552b WatchSource:0}: Error finding container 08aac748590d55431ee9d292cf41e4c40f15953eb628bc556155cffd081c552b: Status 404 returned error can't find the container with id 08aac748590d55431ee9d292cf41e4c40f15953eb628bc556155cffd081c552b
Jul 19 15:20:15.352557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.352525    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:20:15.352639 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.352601    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:20:15.352676 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.352665    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:20:15.522859 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.522810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" event={"ID":"f1514623-f186-4d20-a6d6-26afd74d907a","Type":"ContainerStarted","Data":"9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"}
Jul 19 15:20:15.523056 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.522874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" event={"ID":"f1514623-f186-4d20-a6d6-26afd74d907a","Type":"ContainerStarted","Data":"08aac748590d55431ee9d292cf41e4c40f15953eb628bc556155cffd081c552b"}
Jul 19 15:20:15.540614 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.540576    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5b4ad1c-61dd-49b1-a266-bc0b373f9714" path="/var/lib/kubelet/pods/b5b4ad1c-61dd-49b1-a266-bc0b373f9714/volumes"
Jul 19 15:20:15.542387 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.542334    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podStartSLOduration=1.542317623 podStartE2EDuration="1.542317623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:20:15.541640979 +0000 UTC m=+1996.566650055" watchObservedRunningTime="2026-07-19 15:20:15.542317623 +0000 UTC m=+1996.567326697"
Jul 19 15:20:15.583573 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:15.583530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:16.195140 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.195109    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:16.199768 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.199741    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:16.528631 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.528603    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:16.529476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.529458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:16.888754 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.888650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:16.964647 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.964565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:16.965678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.965643    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"]
Jul 19 15:20:16.969816 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.969794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:16.972400 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.972380    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b0787d8c-serving-cert\""
Jul 19 15:20:16.979090 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:16.979033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"]
Jul 19 15:20:17.028626 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.028597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-maas-api-tls\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.028765 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.028628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7kkz\" (UniqueName: \"kubernetes.io/projected/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-kube-api-access-x7kkz\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.028765 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.028706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-openshift-service-ca\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.040414 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.040383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:17.129431 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.129402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-maas-api-tls\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.129431 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.129431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7kkz\" (UniqueName: \"kubernetes.io/projected/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-kube-api-access-x7kkz\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.129649 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.129480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-openshift-service-ca\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.130140 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.130106    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-openshift-service-ca\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.131782 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.131757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-maas-api-tls\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.138099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.138067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7kkz\" (UniqueName: \"kubernetes.io/projected/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-kube-api-access-x7kkz\") pod \"maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.179468 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.179400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:17.282710 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.282686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:17.415693 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.415664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"]
Jul 19 15:20:17.417411 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:20:17.417382    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5cf5af2_6080_4afb_bbaa_6be53e9f184f.slice/crio-5b175446c59f290baad9f1fdf780c4be09dcc88dbbafb1a02deacc108335ef5d WatchSource:0}: Error finding container 5b175446c59f290baad9f1fdf780c4be09dcc88dbbafb1a02deacc108335ef5d: Status 404 returned error can't find the container with id 5b175446c59f290baad9f1fdf780c4be09dcc88dbbafb1a02deacc108335ef5d
Jul 19 15:20:17.418888 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.418871    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:20:17.538725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.538692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9" event={"ID":"e5cf5af2-6080-4afb-bbaa-6be53e9f184f","Type":"ContainerStarted","Data":"5b175446c59f290baad9f1fdf780c4be09dcc88dbbafb1a02deacc108335ef5d"}
Jul 19 15:20:17.629349 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.629316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:17.729342 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:17.729306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:18.540326 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:18.540284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9" event={"ID":"e5cf5af2-6080-4afb-bbaa-6be53e9f184f","Type":"ContainerStarted","Data":"f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7"}
Jul 19 15:20:18.540729 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:18.540374    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:18.556787 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:18.556744    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9" podStartSLOduration=2.117339379 podStartE2EDuration="2.556729406s" totalImagesPullingTime="439.390027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:20:18.555556687 +0000 UTC m=+1999.580565762" watchObservedRunningTime="2026-07-19 15:20:18.556729406 +0000 UTC m=+1999.581738480"
Jul 19 15:20:19.341096 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.341073    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:20:19.347991 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.347940    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-credential-socket\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.347991 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.347969    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-podinfo\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348000    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-socket\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348019    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istiod-ca-cert\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348048    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66lp9\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-kube-api-access-66lp9\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348094    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-certs\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-envoy\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348176 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348161    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-data\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348486 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-token\") pod \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\" (UID: \"8325ffdb-ebe2-4bae-9bfb-8900d32705b4\") "
Jul 19 15:20:19.348486 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348239    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:20:19.348486 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348358    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:20:19.348486 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:20:19.348725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348517    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:20:19.348725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348609    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.348725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348630    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.348725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348646    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.348725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.348659    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.349154 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.349134    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:20:19.350161 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.350143    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:20:19.350333 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.350312    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:20:19.350633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.350616    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-kube-api-access-66lp9" (OuterVolumeSpecName: "kube-api-access-66lp9") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "kube-api-access-66lp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:20:19.350690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.350625    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8325ffdb-ebe2-4bae-9bfb-8900d32705b4" (UID: "8325ffdb-ebe2-4bae-9bfb-8900d32705b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:20:19.449582 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.449544    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.449582 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.449573    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.449582 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.449585    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.449811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.449596    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.449811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.449607    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66lp9\" (UniqueName: \"kubernetes.io/projected/8325ffdb-ebe2-4bae-9bfb-8900d32705b4-kube-api-access-66lp9\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:19.547678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.547647    2577 generic.go:356] "Generic (PLEG): container finished" podID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerID="e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77" exitCode=0
Jul 19 15:20:19.548092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.547704    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"
Jul 19 15:20:19.548092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.547744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" event={"ID":"8325ffdb-ebe2-4bae-9bfb-8900d32705b4","Type":"ContainerDied","Data":"e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77"}
Jul 19 15:20:19.548092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.547788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq" event={"ID":"8325ffdb-ebe2-4bae-9bfb-8900d32705b4","Type":"ContainerDied","Data":"884f1f4a939394a17a893bca7fc0d969baec6b411e77a4879031b41f20e63b41"}
Jul 19 15:20:19.548092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.547809    2577 scope.go:117] "RemoveContainer" containerID="e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77"
Jul 19 15:20:19.556193 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.556178    2577 scope.go:117] "RemoveContainer" containerID="e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"
Jul 19 15:20:19.563358 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.563333    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"]
Jul 19 15:20:19.564088 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.564060    2577 scope.go:117] "RemoveContainer" containerID="e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77"
Jul 19 15:20:19.564371 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:20:19.564344    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77\": container with ID starting with e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77 not found: ID does not exist" containerID="e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77"
Jul 19 15:20:19.564529 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.564479    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77"} err="failed to get container status \"e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77\": rpc error: code = NotFound desc = could not find container \"e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77\": container with ID starting with e9290c5f5560bdff51da58da8617435027d6a19b28ba4c3afe53d22715e6aa77 not found: ID does not exist"
Jul 19 15:20:19.564529 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.564530    2577 scope.go:117] "RemoveContainer" containerID="e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"
Jul 19 15:20:19.564805 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:20:19.564784    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a\": container with ID starting with e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a not found: ID does not exist" containerID="e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"
Jul 19 15:20:19.564873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.564813    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a"} err="failed to get container status \"e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a\": rpc error: code = NotFound desc = could not find container \"e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a\": container with ID starting with e43fa6982d57721109eff9508a0fb8d87d79dea1fb1f1ddd23133afb14ca488a not found: ID does not exist"
Jul 19 15:20:19.566226 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:19.566208    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6d22b79c-gw-openshift-default-55b7579bdc-cm4jq"]
Jul 19 15:20:21.539427 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:21.539397    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" path="/var/lib/kubelet/pods/8325ffdb-ebe2-4bae-9bfb-8900d32705b4/volumes"
Jul 19 15:20:24.553884 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:24.553850    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:31.529668 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.529631    2577 patch_prober.go:28] interesting pod/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:20:31.530228 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.529687    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:20:31.609432 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.609401    2577 generic.go:356] "Generic (PLEG): container finished" podID="f1514623-f186-4d20-a6d6-26afd74d907a" containerID="9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc" exitCode=0
Jul 19 15:20:31.609595 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.609449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" event={"ID":"f1514623-f186-4d20-a6d6-26afd74d907a","Type":"ContainerDied","Data":"9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"}
Jul 19 15:20:31.609836 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.609818    2577 scope.go:117] "RemoveContainer" containerID="9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"
Jul 19 15:20:31.610828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.610796    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:20:31.610900 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.610885    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:20:31.610937 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:31.610921    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:20:32.617657 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:32.617620    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" event={"ID":"f1514623-f186-4d20-a6d6-26afd74d907a","Type":"ContainerStarted","Data":"806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc"}
Jul 19 15:20:33.195085 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:33.195047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:33.195301 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:33.195277    2577 patch_prober.go:28] interesting pod/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:20:33.195384 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:33.195336    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:20:34.194540 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:34.194483    2577 patch_prober.go:28] interesting pod/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:20:34.194919 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:34.194561    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:20:35.195190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:35.195153    2577 patch_prober.go:28] interesting pod/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:20:35.195627 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:35.195209    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:20:36.194480 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:36.194448    2577 patch_prober.go:28] interesting pod/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:20:36.194670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:36.194520    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:20:37.198375 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:37.198346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:37.198853 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:37.198740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:37.199324 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:37.199306    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:20:42.759112 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759074    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"]
Jul 19 15:20:42.759590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759475    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy"
Jul 19 15:20:42.759590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759487    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy"
Jul 19 15:20:42.759590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759512    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy"
Jul 19 15:20:42.759590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759523    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy"
Jul 19 15:20:42.759590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759588    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy"
Jul 19 15:20:42.759772 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.759597    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8325ffdb-ebe2-4bae-9bfb-8900d32705b4" containerName="istio-proxy"
Jul 19 15:20:42.761787 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.761770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:42.764337 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.764315    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:20:42.771150 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.770784    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"]
Jul 19 15:20:42.862705 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.862670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ac4205-2686-48cd-9aad-8a935e4097d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx\" (UID: \"31ac4205-2686-48cd-9aad-8a935e4097d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:42.963680 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.963646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ac4205-2686-48cd-9aad-8a935e4097d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx\" (UID: \"31ac4205-2686-48cd-9aad-8a935e4097d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:42.964426 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:42.964401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ac4205-2686-48cd-9aad-8a935e4097d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx\" (UID: \"31ac4205-2686-48cd-9aad-8a935e4097d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:43.078338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:43.078252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:43.421948 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:43.421921    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"]
Jul 19 15:20:43.423877 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:20:43.423845    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ac4205_2686_48cd_9aad_8a935e4097d0.slice/crio-0123b9558d2f8d2df9d17a29ebb6606218fbe1a282849d600f79a0124e5ce4da WatchSource:0}: Error finding container 0123b9558d2f8d2df9d17a29ebb6606218fbe1a282849d600f79a0124e5ce4da: Status 404 returned error can't find the container with id 0123b9558d2f8d2df9d17a29ebb6606218fbe1a282849d600f79a0124e5ce4da
Jul 19 15:20:43.680746 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:43.680673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx" event={"ID":"31ac4205-2686-48cd-9aad-8a935e4097d0","Type":"ContainerStarted","Data":"0123b9558d2f8d2df9d17a29ebb6606218fbe1a282849d600f79a0124e5ce4da"}
Jul 19 15:20:44.687174 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:44.687137    2577 generic.go:356] "Generic (PLEG): container finished" podID="31ac4205-2686-48cd-9aad-8a935e4097d0" containerID="816b0089b2f0a466c138067f87f362840238c633327e218dc34377e6475129fe" exitCode=0
Jul 19 15:20:44.687577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:44.687222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx" event={"ID":"31ac4205-2686-48cd-9aad-8a935e4097d0","Type":"ContainerDied","Data":"816b0089b2f0a466c138067f87f362840238c633327e218dc34377e6475129fe"}
Jul 19 15:20:45.826682 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:45.826661    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:45.887419 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:45.887390    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ac4205-2686-48cd-9aad-8a935e4097d0-maas-api-service-ca\") pod \"31ac4205-2686-48cd-9aad-8a935e4097d0\" (UID: \"31ac4205-2686-48cd-9aad-8a935e4097d0\") "
Jul 19 15:20:45.887726 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:45.887705    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ac4205-2686-48cd-9aad-8a935e4097d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "31ac4205-2686-48cd-9aad-8a935e4097d0" (UID: "31ac4205-2686-48cd-9aad-8a935e4097d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:20:45.988914 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:45.988891    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31ac4205-2686-48cd-9aad-8a935e4097d0-maas-api-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:46.699393 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:46.699365    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"
Jul 19 15:20:46.699587 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:46.699393    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx" event={"ID":"31ac4205-2686-48cd-9aad-8a935e4097d0","Type":"ContainerDied","Data":"0123b9558d2f8d2df9d17a29ebb6606218fbe1a282849d600f79a0124e5ce4da"}
Jul 19 15:20:46.699587 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:46.699433    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0123b9558d2f8d2df9d17a29ebb6606218fbe1a282849d600f79a0124e5ce4da"
Jul 19 15:20:51.256365 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:51.256329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:52.678360 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:52.678330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:52.729693 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:52.729665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:53.119122 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.119084    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"]
Jul 19 15:20:53.119367 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.119341    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9" podUID="e5cf5af2-6080-4afb-bbaa-6be53e9f184f" containerName="maas-api" containerID="cri-o://f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7" gracePeriod=30
Jul 19 15:20:53.151711 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.151672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:53.373823 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.373769    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:53.451272 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.451246    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-openshift-service-ca\") pod \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") "
Jul 19 15:20:53.451441 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.451279    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7kkz\" (UniqueName: \"kubernetes.io/projected/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-kube-api-access-x7kkz\") pod \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") "
Jul 19 15:20:53.451441 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.451359    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-maas-api-tls\") pod \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\" (UID: \"e5cf5af2-6080-4afb-bbaa-6be53e9f184f\") "
Jul 19 15:20:53.451667 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.451643    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5cf5af2-6080-4afb-bbaa-6be53e9f184f" (UID: "e5cf5af2-6080-4afb-bbaa-6be53e9f184f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:20:53.453401 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.453368    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5cf5af2-6080-4afb-bbaa-6be53e9f184f" (UID: "e5cf5af2-6080-4afb-bbaa-6be53e9f184f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:20:53.453557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.453414    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-kube-api-access-x7kkz" (OuterVolumeSpecName: "kube-api-access-x7kkz") pod "e5cf5af2-6080-4afb-bbaa-6be53e9f184f" (UID: "e5cf5af2-6080-4afb-bbaa-6be53e9f184f"). InnerVolumeSpecName "kube-api-access-x7kkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:20:53.552759 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.552736    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:53.552759 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.552759    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7kkz\" (UniqueName: \"kubernetes.io/projected/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-kube-api-access-x7kkz\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:53.552897 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.552770    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cf5af2-6080-4afb-bbaa-6be53e9f184f-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:20:53.737590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.737563    2577 generic.go:356] "Generic (PLEG): container finished" podID="e5cf5af2-6080-4afb-bbaa-6be53e9f184f" containerID="f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7" exitCode=0
Jul 19 15:20:53.737972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.737604    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"
Jul 19 15:20:53.737972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.737634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9" event={"ID":"e5cf5af2-6080-4afb-bbaa-6be53e9f184f","Type":"ContainerDied","Data":"f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7"}
Jul 19 15:20:53.737972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.737674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9" event={"ID":"e5cf5af2-6080-4afb-bbaa-6be53e9f184f","Type":"ContainerDied","Data":"5b175446c59f290baad9f1fdf780c4be09dcc88dbbafb1a02deacc108335ef5d"}
Jul 19 15:20:53.737972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.737695    2577 scope.go:117] "RemoveContainer" containerID="f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7"
Jul 19 15:20:53.745929 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.745913    2577 scope.go:117] "RemoveContainer" containerID="f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7"
Jul 19 15:20:53.746160 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:20:53.746141    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7\": container with ID starting with f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7 not found: ID does not exist" containerID="f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7"
Jul 19 15:20:53.746204 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.746169    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7"} err="failed to get container status \"f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7\": rpc error: code = NotFound desc = could not find container \"f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7\": container with ID starting with f88988da1b1fec3fee3ed776f504a26af5633f0b7c26040c093766d6ba2e67a7 not found: ID does not exist"
Jul 19 15:20:53.753326 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.753302    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"]
Jul 19 15:20:53.757975 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:53.757956    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0787d8c-6c6b49b9f4-5vwt9"]
Jul 19 15:20:55.539729 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:55.539702    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5cf5af2-6080-4afb-bbaa-6be53e9f184f" path="/var/lib/kubelet/pods/e5cf5af2-6080-4afb-bbaa-6be53e9f184f/volumes"
Jul 19 15:20:57.825861 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:57.825833    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"]
Jul 19 15:20:57.828455 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:57.828426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:57.828641 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:57.828619    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0787d8c-m44lx"]
Jul 19 15:20:59.053829 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.053792    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"]
Jul 19 15:20:59.054258 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.054067    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy" containerID="cri-o://806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc" gracePeriod=30
Jul 19 15:20:59.068394 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.068366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:59.313874 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.313792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:59.540872 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.540845    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ac4205-2686-48cd-9aad-8a935e4097d0" path="/var/lib/kubelet/pods/31ac4205-2686-48cd-9aad-8a935e4097d0/volumes"
Jul 19 15:20:59.553389 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.553357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:59.853726 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.853694    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"]
Jul 19 15:20:59.854059 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.854047    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cf5af2-6080-4afb-bbaa-6be53e9f184f" containerName="maas-api"
Jul 19 15:20:59.854117 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.854060    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cf5af2-6080-4afb-bbaa-6be53e9f184f" containerName="maas-api"
Jul 19 15:20:59.854117 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.854071    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ac4205-2686-48cd-9aad-8a935e4097d0" containerName="revoke-keys"
Jul 19 15:20:59.854117 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.854077    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ac4205-2686-48cd-9aad-8a935e4097d0" containerName="revoke-keys"
Jul 19 15:20:59.854221 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.854132    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5cf5af2-6080-4afb-bbaa-6be53e9f184f" containerName="maas-api"
Jul 19 15:20:59.854221 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.854142    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ac4205-2686-48cd-9aad-8a935e4097d0" containerName="revoke-keys"
Jul 19 15:20:59.857225 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.857208    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:20:59.859509 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.859474    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-cbf49862-openshift-default-dockercfg-56449\""
Jul 19 15:20:59.866883 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.866860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"]
Jul 19 15:20:59.877373 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:20:59.877309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:00.006093 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-data\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-certs\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-envoy\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-token\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-socket\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006525 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-credential-socket\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006525 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-podinfo\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006525 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istiod-ca-cert\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.006525 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.006385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9snb6\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-kube-api-access-9snb6\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.107756 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-data\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-certs\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-envoy\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-token\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-socket\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-credential-socket\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-podinfo\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istiod-ca-cert\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.107966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9snb6\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-kube-api-access-9snb6\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108212 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.108157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-data\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108707 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.108233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-certs\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108707 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.108471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-socket\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108707 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.108530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-credential-socket\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.108986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.108964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istiod-ca-cert\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.110126 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.110103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-envoy\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.110655 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.110632    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-podinfo\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.116203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.116177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9snb6\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-kube-api-access-9snb6\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.116448 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.116429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-token\") pod \"e2e-derive-cbf49862-openshift-default-8554d96767-vgcht\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") " pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.141671 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.141608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:00.168670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.168647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:00.305415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.305390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"]
Jul 19 15:21:00.306195 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:21:00.306173    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc97b94_f83d_4d6b_890a_963f1ffb6bcd.slice/crio-d1132e8bec0267e5a699e8e752d188a535a1883b120d5bd0ca643397a7c7b048 WatchSource:0}: Error finding container d1132e8bec0267e5a699e8e752d188a535a1883b120d5bd0ca643397a7c7b048: Status 404 returned error can't find the container with id d1132e8bec0267e5a699e8e752d188a535a1883b120d5bd0ca643397a7c7b048
Jul 19 15:21:00.308339 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.308309    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:21:00.308421 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.308369    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:21:00.308421 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.308397    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:21:00.773062 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.773026    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" event={"ID":"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd","Type":"ContainerStarted","Data":"e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"}
Jul 19 15:21:00.773244 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.773094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" event={"ID":"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd","Type":"ContainerStarted","Data":"d1132e8bec0267e5a699e8e752d188a535a1883b120d5bd0ca643397a7c7b048"}
Jul 19 15:21:00.810941 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:00.810895    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" podStartSLOduration=1.810878945 podStartE2EDuration="1.810878945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:21:00.808934562 +0000 UTC m=+2041.833943635" watchObservedRunningTime="2026-07-19 15:21:00.810878945 +0000 UTC m=+2041.835888087"
Jul 19 15:21:01.169401 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:01.169329    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:01.174154 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:01.174131    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:01.777765 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:01.777733    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:01.778545 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:01.778526    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:02.173360 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.173272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:02.276458 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.276427    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"]
Jul 19 15:21:02.279842 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.279819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.282540 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.282479    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-cbf49862-serving-cert\""
Jul 19 15:21:02.295478 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.295455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"]
Jul 19 15:21:02.308004 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.307964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:02.395820 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.395790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:02.431063 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.430961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-maas-api-tls\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.431063 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.430999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q85p9\" (UniqueName: \"kubernetes.io/projected/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-kube-api-access-q85p9\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.431251 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.431126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-openshift-service-ca\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.519029 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.518972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:02.532071 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.532043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-maas-api-tls\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.532215 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.532074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q85p9\" (UniqueName: \"kubernetes.io/projected/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-kube-api-access-q85p9\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.532215 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.532143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-openshift-service-ca\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.532904 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.532884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-openshift-service-ca\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.534702 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.534678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-maas-api-tls\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.539997 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.539979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q85p9\" (UniqueName: \"kubernetes.io/projected/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-kube-api-access-q85p9\") pod \"maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.592299 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.592273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:02.726882 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.726853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"]
Jul 19 15:21:02.728687 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:21:02.728654    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bbb7afa_7776_4185_9ba8_d0a5412ee36d.slice/crio-658037422cf4f1e5ae2f9c252cd7d4934157f8fbf7b5dc8954261155324b6ac9 WatchSource:0}: Error finding container 658037422cf4f1e5ae2f9c252cd7d4934157f8fbf7b5dc8954261155324b6ac9: Status 404 returned error can't find the container with id 658037422cf4f1e5ae2f9c252cd7d4934157f8fbf7b5dc8954261155324b6ac9
Jul 19 15:21:02.784283 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.784252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs" event={"ID":"2bbb7afa-7776-4185-9ba8-d0a5412ee36d","Type":"ContainerStarted","Data":"658037422cf4f1e5ae2f9c252cd7d4934157f8fbf7b5dc8954261155324b6ac9"}
Jul 19 15:21:02.829435 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.829409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:02.928580 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:02.928552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:03.790248 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:03.790210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs" event={"ID":"2bbb7afa-7776-4185-9ba8-d0a5412ee36d","Type":"ContainerStarted","Data":"0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191"}
Jul 19 15:21:03.790651 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:03.790438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:03.807483 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:03.807429    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs" podStartSLOduration=1.366334246 podStartE2EDuration="1.807411642s" totalImagesPullingTime="441.077396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:21:03.805983944 +0000 UTC m=+2044.830993018" watchObservedRunningTime="2026-07-19 15:21:03.807411642 +0000 UTC m=+2044.832420719"
Jul 19 15:21:04.014037 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.013998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:04.119039 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.119009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:04.305415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.305387    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:21:04.452799 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452718    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-certs\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.452799 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452778    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-envoy\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452804    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-socket\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1514623-f186-4d20-a6d6-26afd74d907a-istio-podinfo\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452856    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-istio-token\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452879    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-credential-socket\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452917    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-data\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1514623-f186-4d20-a6d6-26afd74d907a-istiod-ca-cert\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453025 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.452974    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-487k6\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-kube-api-access-487k6\") pod \"f1514623-f186-4d20-a6d6-26afd74d907a\" (UID: \"f1514623-f186-4d20-a6d6-26afd74d907a\") "
Jul 19 15:21:04.453375 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453039    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:04.453375 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453137    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:04.453375 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:04.453375 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453364    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.453610 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453384    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.453610 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453399    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.453610 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453364    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1514623-f186-4d20-a6d6-26afd74d907a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:21:04.453890 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.453869    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:04.455124 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.455097    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:21:04.455308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.455289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1514623-f186-4d20-a6d6-26afd74d907a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:21:04.455390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.455372    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-kube-api-access-487k6" (OuterVolumeSpecName: "kube-api-access-487k6") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "kube-api-access-487k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:21:04.455466 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.455448    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1514623-f186-4d20-a6d6-26afd74d907a" (UID: "f1514623-f186-4d20-a6d6-26afd74d907a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:04.554681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.554652    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.554681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.554676    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1514623-f186-4d20-a6d6-26afd74d907a-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.554681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.554685    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-487k6\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-kube-api-access-487k6\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.554875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.554695    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1514623-f186-4d20-a6d6-26afd74d907a-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.554875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.554703    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1514623-f186-4d20-a6d6-26afd74d907a-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.554875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.554710    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1514623-f186-4d20-a6d6-26afd74d907a-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:04.796399 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.796372    2577 generic.go:356] "Generic (PLEG): container finished" podID="f1514623-f186-4d20-a6d6-26afd74d907a" containerID="806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc" exitCode=0
Jul 19 15:21:04.796774 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.796413    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"
Jul 19 15:21:04.796774 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.796456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" event={"ID":"f1514623-f186-4d20-a6d6-26afd74d907a","Type":"ContainerDied","Data":"806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc"}
Jul 19 15:21:04.796774 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.796520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw" event={"ID":"f1514623-f186-4d20-a6d6-26afd74d907a","Type":"ContainerDied","Data":"08aac748590d55431ee9d292cf41e4c40f15953eb628bc556155cffd081c552b"}
Jul 19 15:21:04.796774 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.796541    2577 scope.go:117] "RemoveContainer" containerID="806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc"
Jul 19 15:21:04.806654 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.806620    2577 scope.go:117] "RemoveContainer" containerID="9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"
Jul 19 15:21:04.814900 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.814882    2577 scope.go:117] "RemoveContainer" containerID="806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc"
Jul 19 15:21:04.815160 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:21:04.815138    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc\": container with ID starting with 806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc not found: ID does not exist" containerID="806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc"
Jul 19 15:21:04.815220 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.815169    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc"} err="failed to get container status \"806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc\": rpc error: code = NotFound desc = could not find container \"806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc\": container with ID starting with 806e0a20f1f50681457ee858f852f2c047fed37754f0282f919ffa4774b317dc not found: ID does not exist"
Jul 19 15:21:04.815220 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.815185    2577 scope.go:117] "RemoveContainer" containerID="9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"
Jul 19 15:21:04.815421 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:21:04.815402    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc\": container with ID starting with 9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc not found: ID does not exist" containerID="9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"
Jul 19 15:21:04.815464 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.815428    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc"} err="failed to get container status \"9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc\": rpc error: code = NotFound desc = could not find container \"9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc\": container with ID starting with 9b4e1e3032536a3ec66ba3ccfe84936ef47a9bf8988282ea40c4804e9f30a7fc not found: ID does not exist"
Jul 19 15:21:04.819894 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.819869    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"]
Jul 19 15:21:04.825610 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.825593    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b0787d8c-openshift-default-c9bf9bf7-plsgw"]
Jul 19 15:21:04.881286 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.881261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:04.956064 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:04.956038    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:05.539265 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:05.539233    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" path="/var/lib/kubelet/pods/f1514623-f186-4d20-a6d6-26afd74d907a/volumes"
Jul 19 15:21:09.801956 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:09.801928    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:13.850168 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:13.850136    2577 generic.go:356] "Generic (PLEG): container finished" podID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerID="e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba" exitCode=0
Jul 19 15:21:13.850591 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:13.850212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" event={"ID":"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd","Type":"ContainerDied","Data":"e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"}
Jul 19 15:21:13.850642 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:13.850595    2577 scope.go:117] "RemoveContainer" containerID="e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"
Jul 19 15:21:13.851577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:13.851550    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:21:13.851655 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:13.851616    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:21:13.851655 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:13.851645    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:21:14.856699 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:14.856667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" event={"ID":"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd","Type":"ContainerStarted","Data":"397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a"}
Jul 19 15:21:15.169068 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:15.168971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:15.170010 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:15.169986    2577 patch_prober.go:28] interesting pod/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:21:15.170143 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:15.170028    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:21:16.169699 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:16.169667    2577 patch_prober.go:28] interesting pod/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:21:16.170083 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:16.169723    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:21:16.778385 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:16.778352    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:17.169911 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:17.169826    2577 patch_prober.go:28] interesting pod/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:21:17.169911 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:17.169892    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:21:18.169346 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:18.169310    2577 patch_prober.go:28] interesting pod/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:21:18.169554 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:18.169365    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:21:19.172515 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:19.172463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:19.173540 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:19.173515    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:27.089316 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089227    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"]
Jul 19 15:21:27.089764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089675    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy"
Jul 19 15:21:27.089764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089690    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy"
Jul 19 15:21:27.089764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089699    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy"
Jul 19 15:21:27.089764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089705    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy"
Jul 19 15:21:27.089764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089766    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy"
Jul 19 15:21:27.089954 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.089777    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1514623-f186-4d20-a6d6-26afd74d907a" containerName="istio-proxy"
Jul 19 15:21:27.092941 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.092923    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:27.095283 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.095260    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:21:27.099794 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.099772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"]
Jul 19 15:21:27.137277 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.137243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cbf49862-btpv9\" (UID: \"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:27.238441 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.238405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cbf49862-btpv9\" (UID: \"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:27.239093 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.239072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cbf49862-btpv9\" (UID: \"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:27.406053 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.405924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:27.537095 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:21:27.537066    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb888bcfb_842b_49a1_97e1_e5fd26a2ddd8.slice/crio-1d3c6f4a3ed9323a5ec30a0879a80bfcd952f7a3eb8af1cdcca53927641ad7fa WatchSource:0}: Error finding container 1d3c6f4a3ed9323a5ec30a0879a80bfcd952f7a3eb8af1cdcca53927641ad7fa: Status 404 returned error can't find the container with id 1d3c6f4a3ed9323a5ec30a0879a80bfcd952f7a3eb8af1cdcca53927641ad7fa
Jul 19 15:21:27.542397 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.542372    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"]
Jul 19 15:21:27.922034 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.922001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9" event={"ID":"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8","Type":"ContainerStarted","Data":"cbaa5e945b2b004c65476217eb3f5f60ee6e3641f2076b29cb57a5766d2046d2"}
Jul 19 15:21:27.922203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.922053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9" event={"ID":"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8","Type":"ContainerStarted","Data":"1d3c6f4a3ed9323a5ec30a0879a80bfcd952f7a3eb8af1cdcca53927641ad7fa"}
Jul 19 15:21:27.937511 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:27.937458    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9" podStartSLOduration=0.937444477 podStartE2EDuration="937.444477ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:21:27.93641725 +0000 UTC m=+2068.961426325" watchObservedRunningTime="2026-07-19 15:21:27.937444477 +0000 UTC m=+2068.962453550"
Jul 19 15:21:28.928155 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:28.928129    2577 generic.go:356] "Generic (PLEG): container finished" podID="b888bcfb-842b-49a1-97e1-e5fd26a2ddd8" containerID="cbaa5e945b2b004c65476217eb3f5f60ee6e3641f2076b29cb57a5766d2046d2" exitCode=0
Jul 19 15:21:28.928565 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:28.928225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9" event={"ID":"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8","Type":"ContainerDied","Data":"cbaa5e945b2b004c65476217eb3f5f60ee6e3641f2076b29cb57a5766d2046d2"}
Jul 19 15:21:30.066784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.066762    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:30.167577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.167551    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8-maas-api-service-ca\") pod \"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8\" (UID: \"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8\") "
Jul 19 15:21:30.167871 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.167852    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b888bcfb-842b-49a1-97e1-e5fd26a2ddd8" (UID: "b888bcfb-842b-49a1-97e1-e5fd26a2ddd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:21:30.268415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.268393    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8-maas-api-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:30.939088 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.939057    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"
Jul 19 15:21:30.939252 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.939108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9" event={"ID":"b888bcfb-842b-49a1-97e1-e5fd26a2ddd8","Type":"ContainerDied","Data":"1d3c6f4a3ed9323a5ec30a0879a80bfcd952f7a3eb8af1cdcca53927641ad7fa"}
Jul 19 15:21:30.939252 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:30.939155    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d3c6f4a3ed9323a5ec30a0879a80bfcd952f7a3eb8af1cdcca53927641ad7fa"
Jul 19 15:21:36.914377 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:36.914350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:37.491282 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.491247    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"]
Jul 19 15:21:37.491537 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.491478    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs" podUID="2bbb7afa-7776-4185-9ba8-d0a5412ee36d" containerName="maas-api" containerID="cri-o://0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191" gracePeriod=30
Jul 19 15:21:37.502843 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.502810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:37.738810 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.738786    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:37.829123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.829056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-openshift-service-ca\") pod \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") "
Jul 19 15:21:37.829123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.829090    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-maas-api-tls\") pod \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") "
Jul 19 15:21:37.829123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.829111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q85p9\" (UniqueName: \"kubernetes.io/projected/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-kube-api-access-q85p9\") pod \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\" (UID: \"2bbb7afa-7776-4185-9ba8-d0a5412ee36d\") "
Jul 19 15:21:37.829530 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.829486    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2bbb7afa-7776-4185-9ba8-d0a5412ee36d" (UID: "2bbb7afa-7776-4185-9ba8-d0a5412ee36d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:21:37.831213 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.831192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2bbb7afa-7776-4185-9ba8-d0a5412ee36d" (UID: "2bbb7afa-7776-4185-9ba8-d0a5412ee36d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:21:37.831317 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.831263    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-kube-api-access-q85p9" (OuterVolumeSpecName: "kube-api-access-q85p9") pod "2bbb7afa-7776-4185-9ba8-d0a5412ee36d" (UID: "2bbb7afa-7776-4185-9ba8-d0a5412ee36d"). InnerVolumeSpecName "kube-api-access-q85p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:21:37.929786 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.929759    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:37.929786 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.929784    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:37.930112 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.929793    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q85p9\" (UniqueName: \"kubernetes.io/projected/2bbb7afa-7776-4185-9ba8-d0a5412ee36d-kube-api-access-q85p9\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:37.979484 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.979454    2577 generic.go:356] "Generic (PLEG): container finished" podID="2bbb7afa-7776-4185-9ba8-d0a5412ee36d" containerID="0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191" exitCode=0
Jul 19 15:21:37.979633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.979517    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs" event={"ID":"2bbb7afa-7776-4185-9ba8-d0a5412ee36d","Type":"ContainerDied","Data":"0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191"}
Jul 19 15:21:37.979633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.979544    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"
Jul 19 15:21:37.979633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.979615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs" event={"ID":"2bbb7afa-7776-4185-9ba8-d0a5412ee36d","Type":"ContainerDied","Data":"658037422cf4f1e5ae2f9c252cd7d4934157f8fbf7b5dc8954261155324b6ac9"}
Jul 19 15:21:37.979764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.979649    2577 scope.go:117] "RemoveContainer" containerID="0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191"
Jul 19 15:21:37.988678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.988662    2577 scope.go:117] "RemoveContainer" containerID="0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191"
Jul 19 15:21:37.988928 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:21:37.988912    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191\": container with ID starting with 0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191 not found: ID does not exist" containerID="0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191"
Jul 19 15:21:37.988966 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.988937    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191"} err="failed to get container status \"0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191\": rpc error: code = NotFound desc = could not find container \"0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191\": container with ID starting with 0c5db287fe0ded36a0784d48dd72e187f22ba90804bb02d17f8dc326d7426191 not found: ID does not exist"
Jul 19 15:21:37.999838 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:37.999815    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"]
Jul 19 15:21:38.003877 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:38.003857    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cbf49862-7bbbd878bd-zt6cs"]
Jul 19 15:21:39.539730 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:39.539699    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bbb7afa-7776-4185-9ba8-d0a5412ee36d" path="/var/lib/kubelet/pods/2bbb7afa-7776-4185-9ba8-d0a5412ee36d/volumes"
Jul 19 15:21:42.154665 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.154624    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"]
Jul 19 15:21:42.156405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.156386    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cbf49862-btpv9"]
Jul 19 15:21:42.160715 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.160685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:42.299753 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.299723    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"]
Jul 19 15:21:42.299976 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.299952    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy" containerID="cri-o://397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a" gracePeriod=30
Jul 19 15:21:42.321166 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.321138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:42.570366 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:42.570337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:43.070091 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:43.070049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:43.170727 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:43.170692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:43.273671 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:43.273634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:43.539748 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:43.539720    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b888bcfb-842b-49a1-97e1-e5fd26a2ddd8" path="/var/lib/kubelet/pods/b888bcfb-842b-49a1-97e1-e5fd26a2ddd8/volumes"
Jul 19 15:21:44.767553 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:44.767516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:47.550135 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.550112    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:47.605700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-data\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-token\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605681    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-socket\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-podinfo\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-credential-socket\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-envoy\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605821    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9snb6\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-kube-api-access-9snb6\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-certs\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.605950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.605907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istiod-ca-cert\") pod \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\" (UID: \"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd\") "
Jul 19 15:21:47.606266 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:47.606327 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606285    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:47.606399 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606377    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:21:47.606454 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606402    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-data" (OuterVolumeSpecName: "istio-data") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:47.606822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606657    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:47.606822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606740    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.606822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606758    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.606822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606774    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.606822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606787    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.606822 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.606799    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.608019 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.607993    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:21:47.608129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.608061    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-token" (OuterVolumeSpecName: "istio-token") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:21:47.608275 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.608258    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:21:47.608705 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.608684    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-kube-api-access-9snb6" (OuterVolumeSpecName: "kube-api-access-9snb6") pod "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" (UID: "4dc97b94-f83d-4d6b-890a-963f1ffb6bcd"). InnerVolumeSpecName "kube-api-access-9snb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:21:47.707823 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.707798    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.707823 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.707820    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.707981 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.707829    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:47.707981 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:47.707839    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9snb6\" (UniqueName: \"kubernetes.io/projected/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd-kube-api-access-9snb6\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:21:48.031545 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.031510    2577 generic.go:356] "Generic (PLEG): container finished" podID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerID="397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a" exitCode=0
Jul 19 15:21:48.031717 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.031571    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"
Jul 19 15:21:48.031717 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.031597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" event={"ID":"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd","Type":"ContainerDied","Data":"397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a"}
Jul 19 15:21:48.031717 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.031654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht" event={"ID":"4dc97b94-f83d-4d6b-890a-963f1ffb6bcd","Type":"ContainerDied","Data":"d1132e8bec0267e5a699e8e752d188a535a1883b120d5bd0ca643397a7c7b048"}
Jul 19 15:21:48.031717 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.031677    2577 scope.go:117] "RemoveContainer" containerID="397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a"
Jul 19 15:21:48.040824 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.040807    2577 scope.go:117] "RemoveContainer" containerID="e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"
Jul 19 15:21:48.048764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.048746    2577 scope.go:117] "RemoveContainer" containerID="397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a"
Jul 19 15:21:48.049007 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:21:48.048990    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a\": container with ID starting with 397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a not found: ID does not exist" containerID="397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a"
Jul 19 15:21:48.049068 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.049016    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a"} err="failed to get container status \"397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a\": rpc error: code = NotFound desc = could not find container \"397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a\": container with ID starting with 397c0582cb214edd0ef7049e0a4a334649bce8a6f9e15ecf817f777bc010770a not found: ID does not exist"
Jul 19 15:21:48.049068 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.049034    2577 scope.go:117] "RemoveContainer" containerID="e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"
Jul 19 15:21:48.049240 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:21:48.049224    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba\": container with ID starting with e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba not found: ID does not exist" containerID="e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"
Jul 19 15:21:48.049283 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.049246    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba"} err="failed to get container status \"e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba\": rpc error: code = NotFound desc = could not find container \"e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba\": container with ID starting with e966379c7c2b724ca2209d7a34f59bbafa5532e466eaab4f4b3c07049b8d53ba not found: ID does not exist"
Jul 19 15:21:48.052090 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.052070    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"]
Jul 19 15:21:48.058114 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:48.058092    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-cbf49862-openshift-default-8554d96767-vgcht"]
Jul 19 15:21:49.540035 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:49.539998    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" path="/var/lib/kubelet/pods/4dc97b94-f83d-4d6b-890a-963f1ffb6bcd/volumes"
Jul 19 15:21:50.707184 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:50.707150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:50.811985 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:50.811949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:50.907353 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:50.907325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:51.007429 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:51.007393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:51.442021 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:21:51.441943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:17.448249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:17.448215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:23.710971 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:23.710914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:23.818356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:23.818317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:23.947472 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:23.947432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:39.852977 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:39.852940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:45.778009 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:45.777968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:45.840851 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:45.840815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:45.910667 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:45.910634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:46.009892 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:46.009863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:56.555672 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:56.555589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:57.393637 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:22:57.393607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:03.515801 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:03.515761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:03.628863 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:03.628809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:03.717433 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:03.717388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:03.817215 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:03.817129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:04.649904 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:04.649865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:04.750597 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:04.750555    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:04.849604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:04.849572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:06.513568 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:06.513519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:12.573295 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:12.573256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:12.674308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:12.674276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:13.266298 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:13.266262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:19.233066 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.233029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:19.332871 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.332832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:19.477608 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.477573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:19.792216 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792170    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"]
Jul 19 15:23:19.792549 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792533    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy"
Jul 19 15:23:19.792549 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792548    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy"
Jul 19 15:23:19.792688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792563    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b888bcfb-842b-49a1-97e1-e5fd26a2ddd8" containerName="revoke-keys"
Jul 19 15:23:19.792688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792568    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b888bcfb-842b-49a1-97e1-e5fd26a2ddd8" containerName="revoke-keys"
Jul 19 15:23:19.792688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792583    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbb7afa-7776-4185-9ba8-d0a5412ee36d" containerName="maas-api"
Jul 19 15:23:19.792688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792588    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbb7afa-7776-4185-9ba8-d0a5412ee36d" containerName="maas-api"
Jul 19 15:23:19.792688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792668    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bbb7afa-7776-4185-9ba8-d0a5412ee36d" containerName="maas-api"
Jul 19 15:23:19.792688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792680    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b888bcfb-842b-49a1-97e1-e5fd26a2ddd8" containerName="revoke-keys"
Jul 19 15:23:19.792988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792691    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy"
Jul 19 15:23:19.792988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.792699    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy"
Jul 19 15:23:19.795722 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.795697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.798414 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.798390    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f1f62af9-openshift-default-dockercfg-bndht\""
Jul 19 15:23:19.817181 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.817156    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"]
Jul 19 15:23:19.826010 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.825978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:19.869748 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.869712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-socket\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.869748 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.869752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istiod-ca-cert\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870007 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.869773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-data\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870007 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.869866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-podinfo\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870007 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.869912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-584m4\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-kube-api-access-584m4\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870007 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.869978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-envoy\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870244 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.870007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-certs\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870244 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.870058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-token\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.870244 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.870084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-credential-socket\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-socket\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istiod-ca-cert\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971394 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-data\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971394 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-podinfo\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971394 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-584m4\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-kube-api-access-584m4\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-envoy\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-certs\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-token\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-socket\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.971577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.971541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-credential-socket\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.972183 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.972156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istiod-ca-cert\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.972300 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.972187    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-certs\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.972300 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.972253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-credential-socket\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.972400 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.972333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-data\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.974082 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.974058    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-envoy\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.974082 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.974073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-podinfo\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.979520 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.979465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-token\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:19.979664 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:19.979647    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-584m4\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-kube-api-access-584m4\") pod \"e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") " pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:20.041192 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.041158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:20.108580 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.108465    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:20.257434 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.257408    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"]
Jul 19 15:23:20.259018 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:23:20.258989    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d396b5b_6c94_47e1_8ec7_b13b54bb4ea4.slice/crio-d1deb1258685fc7ce79cd4da194938caf7fd83c38216c3c832e67376f4d3421d WatchSource:0}: Error finding container d1deb1258685fc7ce79cd4da194938caf7fd83c38216c3c832e67376f4d3421d: Status 404 returned error can't find the container with id d1deb1258685fc7ce79cd4da194938caf7fd83c38216c3c832e67376f4d3421d
Jul 19 15:23:20.261099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.261066    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:23:20.261198 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.261149    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:23:20.261198 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.261178    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:23:20.489311 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.489276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" event={"ID":"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4","Type":"ContainerStarted","Data":"74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"}
Jul 19 15:23:20.489476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.489333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" event={"ID":"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4","Type":"ContainerStarted","Data":"d1deb1258685fc7ce79cd4da194938caf7fd83c38216c3c832e67376f4d3421d"}
Jul 19 15:23:20.508469 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:20.508421    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podStartSLOduration=1.5084041419999998 podStartE2EDuration="1.508404142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:23:20.507437742 +0000 UTC m=+2181.532446816" watchObservedRunningTime="2026-07-19 15:23:20.508404142 +0000 UTC m=+2181.533413217"
Jul 19 15:23:21.109512 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:21.109466    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:21.114551 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:21.114487    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:21.494718 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:21.494692    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:21.495581 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:21.495562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:22.692743 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.692694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:22.808692 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.808655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:22.849473 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.849438    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"]
Jul 19 15:23:22.850016 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.850001    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy"
Jul 19 15:23:22.850069 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.850020    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc97b94-f83d-4d6b-890a-963f1ffb6bcd" containerName="istio-proxy"
Jul 19 15:23:22.853594 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.853568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:22.856986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.856958    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f1f62af9-serving-cert\""
Jul 19 15:23:22.876289 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.876263    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"]
Jul 19 15:23:22.900597 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.900571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:22.900741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.900608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wzk9\" (UniqueName: \"kubernetes.io/projected/3c0d1887-a2dd-4172-816a-3b4566687269-kube-api-access-7wzk9\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:22.900741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.900644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c0d1887-a2dd-4172-816a-3b4566687269-openshift-service-ca\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:22.960956 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:22.960870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:23.001379 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.001348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.001379 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.001382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wzk9\" (UniqueName: \"kubernetes.io/projected/3c0d1887-a2dd-4172-816a-3b4566687269-kube-api-access-7wzk9\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.001687 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.001405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c0d1887-a2dd-4172-816a-3b4566687269-openshift-service-ca\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.001687 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:23:23.001535    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-serving-cert: secret "maas-api-e2e-mt-f1f62af9-serving-cert" not found
Jul 19 15:23:23.001687 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:23:23.001606    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls podName:3c0d1887-a2dd-4172-816a-3b4566687269 nodeName:}" failed. No retries permitted until 2026-07-19 15:23:23.50158671 +0000 UTC m=+2184.526595765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls") pod "maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" (UID: "3c0d1887-a2dd-4172-816a-3b4566687269") : secret "maas-api-e2e-mt-f1f62af9-serving-cert" not found
Jul 19 15:23:23.002017 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.001999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c0d1887-a2dd-4172-816a-3b4566687269-openshift-service-ca\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.012485 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.012458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wzk9\" (UniqueName: \"kubernetes.io/projected/3c0d1887-a2dd-4172-816a-3b4566687269-kube-api-access-7wzk9\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.249647 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.249606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:23.450420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.450386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:23.506159 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.506061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.508886 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.508856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls\") pod \"maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.549530 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.549480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:23.766433 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.766341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:23.920468 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:23.920436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"]
Jul 19 15:23:23.923953 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:23:23.923925    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0d1887_a2dd_4172_816a_3b4566687269.slice/crio-a5511091d89bc6ccd2ac39c67cfd5594830570c67cd7b4976d822fb27b023d63 WatchSource:0}: Error finding container a5511091d89bc6ccd2ac39c67cfd5594830570c67cd7b4976d822fb27b023d63: Status 404 returned error can't find the container with id a5511091d89bc6ccd2ac39c67cfd5594830570c67cd7b4976d822fb27b023d63
Jul 19 15:23:24.514475 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:24.514435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" event={"ID":"3c0d1887-a2dd-4172-816a-3b4566687269","Type":"ContainerStarted","Data":"a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0"}
Jul 19 15:23:24.514700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:24.514518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" event={"ID":"3c0d1887-a2dd-4172-816a-3b4566687269","Type":"ContainerStarted","Data":"a5511091d89bc6ccd2ac39c67cfd5594830570c67cd7b4976d822fb27b023d63"}
Jul 19 15:23:24.514700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:24.514550    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:24.532226 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:24.532176    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" podStartSLOduration=2.111470585 podStartE2EDuration="2.532159141s" totalImagesPullingTime="420.688556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:23:24.531983381 +0000 UTC m=+2185.556992456" watchObservedRunningTime="2026-07-19 15:23:24.532159141 +0000 UTC m=+2185.557168215"
Jul 19 15:23:27.214338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:27.214302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:27.925566 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:27.925527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:28.046552 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:28.046486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:28.387007 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:28.386966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:30.534783 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:30.534746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:34.121136 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.121095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:34.222622 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.222581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:34.476583 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.476552    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"]
Jul 19 15:23:34.480437 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.480414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:34.482930 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.482910    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:23:34.487557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.487532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"]
Jul 19 15:23:34.610933 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.610890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f1f62af9-jplln\" (UID: \"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:34.712360 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.712321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f1f62af9-jplln\" (UID: \"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:34.713040 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.713013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f1f62af9-jplln\" (UID: \"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:34.794816 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.794728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:34.942397 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:34.942368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"]
Jul 19 15:23:34.944418 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:23:34.944390    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e79e063_ed1c_4a0b_92cf_bfe59a66b9ee.slice/crio-ab48b71a129d8e080e9e5924f0bc02d1d549b6dc789f3595114621aaba72cf93 WatchSource:0}: Error finding container ab48b71a129d8e080e9e5924f0bc02d1d549b6dc789f3595114621aaba72cf93: Status 404 returned error can't find the container with id ab48b71a129d8e080e9e5924f0bc02d1d549b6dc789f3595114621aaba72cf93
Jul 19 15:23:35.570810 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:35.570769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln" event={"ID":"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee","Type":"ContainerStarted","Data":"3d4271c4af1fa5d7b012ed93f59ddb910bf71f4915b6720e0fd2b2e3f3b2b89e"}
Jul 19 15:23:35.571209 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:35.570835    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln" event={"ID":"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee","Type":"ContainerStarted","Data":"ab48b71a129d8e080e9e5924f0bc02d1d549b6dc789f3595114621aaba72cf93"}
Jul 19 15:23:35.585810 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:35.585759    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln" podStartSLOduration=1.585744002 podStartE2EDuration="1.585744002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:23:35.584364341 +0000 UTC m=+2196.609373417" watchObservedRunningTime="2026-07-19 15:23:35.585744002 +0000 UTC m=+2196.610753077"
Jul 19 15:23:36.496040 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.496009    2577 patch_prober.go:28] interesting pod/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:23:36.496245 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.496064    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:23:36.576889 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.576856    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee" containerID="3d4271c4af1fa5d7b012ed93f59ddb910bf71f4915b6720e0fd2b2e3f3b2b89e" exitCode=0
Jul 19 15:23:36.577331 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.576943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln" event={"ID":"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee","Type":"ContainerDied","Data":"3d4271c4af1fa5d7b012ed93f59ddb910bf71f4915b6720e0fd2b2e3f3b2b89e"}
Jul 19 15:23:36.578443 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.578423    2577 generic.go:356] "Generic (PLEG): container finished" podID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerID="74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184" exitCode=0
Jul 19 15:23:36.578557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.578514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" event={"ID":"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4","Type":"ContainerDied","Data":"74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"}
Jul 19 15:23:36.578879 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.578866    2577 scope.go:117] "RemoveContainer" containerID="74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"
Jul 19 15:23:36.579897 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.579869    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:23:36.579962 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.579947    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:23:36.579999 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:36.579983    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:23:37.585338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:37.585307    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" event={"ID":"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4","Type":"ContainerStarted","Data":"2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687"}
Jul 19 15:23:37.759711 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:37.759683    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:37.845873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:37.845779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee-maas-api-service-ca\") pod \"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee\" (UID: \"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee\") "
Jul 19 15:23:37.846127 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:37.846101    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee" (UID: "4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:23:37.946478 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:37.946444    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee-maas-api-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:23:38.109191 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:38.109097    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:38.109347 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:38.109265    2577 patch_prober.go:28] interesting pod/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:23:38.109347 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:38.109310    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:23:38.591053 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:38.591020    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"
Jul 19 15:23:38.591444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:38.591055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln" event={"ID":"4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee","Type":"ContainerDied","Data":"ab48b71a129d8e080e9e5924f0bc02d1d549b6dc789f3595114621aaba72cf93"}
Jul 19 15:23:38.591444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:38.591106    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab48b71a129d8e080e9e5924f0bc02d1d549b6dc789f3595114621aaba72cf93"
Jul 19 15:23:39.109712 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:39.109671    2577 patch_prober.go:28] interesting pod/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:23:39.109885 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:39.109732    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:23:40.109584 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:40.109544    2577 patch_prober.go:28] interesting pod/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:23:40.109949 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:40.109601    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:23:41.109013 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:41.108977    2577 patch_prober.go:28] interesting pod/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:23:41.109202 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:41.109032    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:23:42.112892 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:42.112861    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:42.113280 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:42.113212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:42.113811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:42.113793    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:23:44.967761 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:44.967726    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"]
Jul 19 15:23:44.968264 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:44.967951    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" podUID="3c0d1887-a2dd-4172-816a-3b4566687269" containerName="maas-api" containerID="cri-o://a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0" gracePeriod=30
Jul 19 15:23:44.991972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:44.991943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:45.224795 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.224771    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:45.321912 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.321884    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wzk9\" (UniqueName: \"kubernetes.io/projected/3c0d1887-a2dd-4172-816a-3b4566687269-kube-api-access-7wzk9\") pod \"3c0d1887-a2dd-4172-816a-3b4566687269\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") "
Jul 19 15:23:45.322065 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.321976    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c0d1887-a2dd-4172-816a-3b4566687269-openshift-service-ca\") pod \"3c0d1887-a2dd-4172-816a-3b4566687269\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") "
Jul 19 15:23:45.322065 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.322000    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls\") pod \"3c0d1887-a2dd-4172-816a-3b4566687269\" (UID: \"3c0d1887-a2dd-4172-816a-3b4566687269\") "
Jul 19 15:23:45.322373 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.322345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c0d1887-a2dd-4172-816a-3b4566687269-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c0d1887-a2dd-4172-816a-3b4566687269" (UID: "3c0d1887-a2dd-4172-816a-3b4566687269"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:23:45.323979 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.323949    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c0d1887-a2dd-4172-816a-3b4566687269" (UID: "3c0d1887-a2dd-4172-816a-3b4566687269"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:23:45.324055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.324033    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0d1887-a2dd-4172-816a-3b4566687269-kube-api-access-7wzk9" (OuterVolumeSpecName: "kube-api-access-7wzk9") pod "3c0d1887-a2dd-4172-816a-3b4566687269" (UID: "3c0d1887-a2dd-4172-816a-3b4566687269"). InnerVolumeSpecName "kube-api-access-7wzk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:23:45.423745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.423716    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c0d1887-a2dd-4172-816a-3b4566687269-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:23:45.423745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.423741    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c0d1887-a2dd-4172-816a-3b4566687269-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:23:45.423745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.423750    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wzk9\" (UniqueName: \"kubernetes.io/projected/3c0d1887-a2dd-4172-816a-3b4566687269-kube-api-access-7wzk9\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:23:45.626826 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.626795    2577 generic.go:356] "Generic (PLEG): container finished" podID="3c0d1887-a2dd-4172-816a-3b4566687269" containerID="a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0" exitCode=0
Jul 19 15:23:45.627004 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.626836    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"
Jul 19 15:23:45.627004 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.626875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" event={"ID":"3c0d1887-a2dd-4172-816a-3b4566687269","Type":"ContainerDied","Data":"a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0"}
Jul 19 15:23:45.627004 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.626925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn" event={"ID":"3c0d1887-a2dd-4172-816a-3b4566687269","Type":"ContainerDied","Data":"a5511091d89bc6ccd2ac39c67cfd5594830570c67cd7b4976d822fb27b023d63"}
Jul 19 15:23:45.627004 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.626958    2577 scope.go:117] "RemoveContainer" containerID="a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0"
Jul 19 15:23:45.635809 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.635791    2577 scope.go:117] "RemoveContainer" containerID="a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0"
Jul 19 15:23:45.636082 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:23:45.636062    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0\": container with ID starting with a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0 not found: ID does not exist" containerID="a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0"
Jul 19 15:23:45.636155 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.636090    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0"} err="failed to get container status \"a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0\": rpc error: code = NotFound desc = could not find container \"a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0\": container with ID starting with a89ae0fa12d3781304546b742745521c52b658d67784fdec5c739c92dab60fe0 not found: ID does not exist"
Jul 19 15:23:45.643336 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.643311    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"]
Jul 19 15:23:45.646671 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:45.646650    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f1f62af9-5654996f7c-sfxjn"]
Jul 19 15:23:47.539083 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:47.539053    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0d1887-a2dd-4172-816a-3b4566687269" path="/var/lib/kubelet/pods/3c0d1887-a2dd-4172-816a-3b4566687269/volumes"
Jul 19 15:23:49.540927 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:49.540899    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"]
Jul 19 15:23:49.543869 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:49.543846    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f1f62af9-jplln"]
Jul 19 15:23:49.546235 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:49.546207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:49.933396 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:49.933309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:51.540546 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:51.540514    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee" path="/var/lib/kubelet/pods/4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee/volumes"
Jul 19 15:23:55.880813 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:55.880778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:55.947777 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:55.947728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:56.144086 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.144013    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"]
Jul 19 15:23:56.144278 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.144259    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" containerID="cri-o://2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687" gracePeriod=30
Jul 19 15:23:56.164737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.164694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:56.399951 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.399864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:56.527833 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.527776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:56.629840 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.629805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:56.751469 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:56.751437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:57.115940 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:57.115845    2577 patch_prober.go:28] interesting pod/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 19 15:23:57.115940 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:23:57.115924    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:24:01.392161 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.392136    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:24:01.557272 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557244    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-socket\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557285    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-certs\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557333    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istiod-ca-cert\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-podinfo\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557370    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-token\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-data\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-envoy\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-584m4\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-kube-api-access-584m4\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557635    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-credential-socket\") pod \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\" (UID: \"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4\") "
Jul 19 15:24:01.557779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:01.557779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557770    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:24:01.557993 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557946    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:01.558046 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.557998    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:01.558046 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.558023    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.558046 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.558039    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.558201 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.558049    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.558358 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.558326    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:01.559763 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.559726    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:24:01.559875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.559795    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:01.559875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.559826    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-kube-api-access-584m4" (OuterVolumeSpecName: "kube-api-access-584m4") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "kube-api-access-584m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:24:01.559875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.559855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" (UID: "7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:24:01.659548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.659489    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.659548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.659541    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.659548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.659552    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.659548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.659560    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.659866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.659569    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.659866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.659577    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-584m4\" (UniqueName: \"kubernetes.io/projected/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4-kube-api-access-584m4\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:24:01.707264 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.707235    2577 generic.go:356] "Generic (PLEG): container finished" podID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerID="2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687" exitCode=0
Jul 19 15:24:01.707435 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.707284    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"
Jul 19 15:24:01.707435 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.707322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" event={"ID":"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4","Type":"ContainerDied","Data":"2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687"}
Jul 19 15:24:01.707435 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.707382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk" event={"ID":"7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4","Type":"ContainerDied","Data":"d1deb1258685fc7ce79cd4da194938caf7fd83c38216c3c832e67376f4d3421d"}
Jul 19 15:24:01.707435 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.707412    2577 scope.go:117] "RemoveContainer" containerID="2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687"
Jul 19 15:24:01.718540 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.718517    2577 scope.go:117] "RemoveContainer" containerID="74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"
Jul 19 15:24:01.726244 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.726226    2577 scope.go:117] "RemoveContainer" containerID="2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687"
Jul 19 15:24:01.726510 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:24:01.726474    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687\": container with ID starting with 2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687 not found: ID does not exist" containerID="2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687"
Jul 19 15:24:01.726570 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.726520    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687"} err="failed to get container status \"2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687\": rpc error: code = NotFound desc = could not find container \"2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687\": container with ID starting with 2b194a548775404cf2a8eb19cde4281eb38834474ed571794bb4e0296850f687 not found: ID does not exist"
Jul 19 15:24:01.726570 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.726539    2577 scope.go:117] "RemoveContainer" containerID="74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"
Jul 19 15:24:01.726796 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:24:01.726778    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184\": container with ID starting with 74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184 not found: ID does not exist" containerID="74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"
Jul 19 15:24:01.726837 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.726804    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184"} err="failed to get container status \"74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184\": rpc error: code = NotFound desc = could not find container \"74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184\": container with ID starting with 74836119c590040b3c63b0a7befb0229d1c0105c4222976c24e42bddd9fe6184 not found: ID does not exist"
Jul 19 15:24:01.729991 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.729971    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"]
Jul 19 15:24:01.734855 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:01.734836    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f1f62af9-openshift-default-5f85b44f67-vdjnk"]
Jul 19 15:24:02.358027 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:02.357996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:02.465676 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:02.465643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:02.595199 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:02.595170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:02.752265 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:02.752233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:03.539138 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:03.539111    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" path="/var/lib/kubelet/pods/7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4/volumes"
Jul 19 15:24:08.391317 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:08.391281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:08.452876 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:08.452838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:08.991972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:08.991937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:09.040939 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:09.040911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:15.319775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.319743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:15.616453 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616375    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"]
Jul 19 15:24:15.616924 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616908    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy"
Jul 19 15:24:15.616972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616927    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy"
Jul 19 15:24:15.616972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616952    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0d1887-a2dd-4172-816a-3b4566687269" containerName="maas-api"
Jul 19 15:24:15.616972 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616961    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0d1887-a2dd-4172-816a-3b4566687269" containerName="maas-api"
Jul 19 15:24:15.617067 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616986    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee" containerName="revoke-keys"
Jul 19 15:24:15.617067 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.616995    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee" containerName="revoke-keys"
Jul 19 15:24:15.617129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.617074    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy"
Jul 19 15:24:15.617129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.617086    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy"
Jul 19 15:24:15.617129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.617094    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e79e063-ed1c-4a0b-92cf-bfe59a66b9ee" containerName="revoke-keys"
Jul 19 15:24:15.617129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.617107    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0d1887-a2dd-4172-816a-3b4566687269" containerName="maas-api"
Jul 19 15:24:15.620392 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.620370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.622782 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.622762    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-28015a4b-openshift-default-dockercfg-km8l4\""
Jul 19 15:24:15.630206 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.630185    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"]
Jul 19 15:24:15.638773 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.638748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:15.675249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-podinfo\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-data\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-token\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istiod-ca-cert\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-credential-socket\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-certs\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkr7z\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-kube-api-access-xkr7z\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675726 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-envoy\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.675726 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.675518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-socket\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776520 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-certs\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkr7z\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-kube-api-access-xkr7z\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-envoy\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-socket\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-podinfo\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-data\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.776686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-token\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777037 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istiod-ca-cert\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777037 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.776750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-credential-socket\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777162 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.777106    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-certs\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777162 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.777131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-credential-socket\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777275 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.777162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-socket\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.777314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-data\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.777557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.777536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istiod-ca-cert\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.779449 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.779426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:15.779562 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.779461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-envoy\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.779718 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.779700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-podinfo\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.784171 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.784150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-token\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.784398 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.784381    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkr7z\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-kube-api-access-xkr7z\") pod \"e2e-mt-28015a4b-openshift-default-6867f46479-djrpc\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") " pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:15.932839 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:15.932763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:16.070086 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.070064    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"]
Jul 19 15:24:16.072234 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:24:16.072208    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2171d49a_7b04_4a81_81b0_84905bfd0ab6.slice/crio-e783113ebce2e452f1d8b7ae986782c9b90271b825bc986229094718b56c6b2d WatchSource:0}: Error finding container e783113ebce2e452f1d8b7ae986782c9b90271b825bc986229094718b56c6b2d: Status 404 returned error can't find the container with id e783113ebce2e452f1d8b7ae986782c9b90271b825bc986229094718b56c6b2d
Jul 19 15:24:16.074383 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.074347    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:24:16.074474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.074433    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:24:16.074561 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.074478    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:24:16.781942 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.781907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" event={"ID":"2171d49a-7b04-4a81-81b0-84905bfd0ab6","Type":"ContainerStarted","Data":"076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"}
Jul 19 15:24:16.782370 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.781966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" event={"ID":"2171d49a-7b04-4a81-81b0-84905bfd0ab6","Type":"ContainerStarted","Data":"e783113ebce2e452f1d8b7ae986782c9b90271b825bc986229094718b56c6b2d"}
Jul 19 15:24:16.801221 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.801173    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podStartSLOduration=1.801158387 podStartE2EDuration="1.801158387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:24:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:24:16.799667419 +0000 UTC m=+2237.824676494" watchObservedRunningTime="2026-07-19 15:24:16.801158387 +0000 UTC m=+2237.826167462"
Jul 19 15:24:16.933035 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:16.933008    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:17.938269 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:17.938238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:18.728390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.728358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:18.792775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.792745    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:18.793626 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.793609    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:18.812566 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.812537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:18.821292 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.821263    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"]
Jul 19 15:24:18.821767 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.821736    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy"
Jul 19 15:24:18.821767 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.821768    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d396b5b-6c94-47e1-8ec7-b13b54bb4ea4" containerName="istio-proxy"
Jul 19 15:24:18.824880 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.824859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:18.827527 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.827477    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-28015a4b-serving-cert\""
Jul 19 15:24:18.835175 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.835146    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"]
Jul 19 15:24:18.887421 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.887384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:18.904552 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.904513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9m8t\" (UniqueName: \"kubernetes.io/projected/9f098845-af24-43a7-ae3c-b815efba60cb-kube-api-access-t9m8t\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:18.904678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.904659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:18.904757 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:18.904725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f098845-af24-43a7-ae3c-b815efba60cb-openshift-service-ca\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.006208 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.006117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.006208 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.006167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f098845-af24-43a7-ae3c-b815efba60cb-openshift-service-ca\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.006208 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.006211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9m8t\" (UniqueName: \"kubernetes.io/projected/9f098845-af24-43a7-ae3c-b815efba60cb-kube-api-access-t9m8t\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.006695 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:24:19.006277    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-serving-cert: secret "maas-api-e2e-mt-28015a4b-serving-cert" not found
Jul 19 15:24:19.006695 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:24:19.006362    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls podName:9f098845-af24-43a7-ae3c-b815efba60cb nodeName:}" failed. No retries permitted until 2026-07-19 15:24:19.506338249 +0000 UTC m=+2240.531347310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls") pod "maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" (UID: "9f098845-af24-43a7-ae3c-b815efba60cb") : secret "maas-api-e2e-mt-28015a4b-serving-cert" not found
Jul 19 15:24:19.006870 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.006852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f098845-af24-43a7-ae3c-b815efba60cb-openshift-service-ca\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.016264 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.016237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9m8t\" (UniqueName: \"kubernetes.io/projected/9f098845-af24-43a7-ae3c-b815efba60cb-kube-api-access-t9m8t\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.194278 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.194250    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:19.392165 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.392061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:19.492335 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.492289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:19.510404 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.510380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.512777 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.512753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls\") pod \"maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.740428 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.740396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:19.875704 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:19.875676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"]
Jul 19 15:24:19.877345 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:24:19.877318    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f098845_af24_43a7_ae3c_b815efba60cb.slice/crio-ab55be6559c635d6f81ec38ff73bacdc0fc9c47d59f50c89b51cf2bba2f0dbbc WatchSource:0}: Error finding container ab55be6559c635d6f81ec38ff73bacdc0fc9c47d59f50c89b51cf2bba2f0dbbc: Status 404 returned error can't find the container with id ab55be6559c635d6f81ec38ff73bacdc0fc9c47d59f50c89b51cf2bba2f0dbbc
Jul 19 15:24:20.806280 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:20.806239    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" event={"ID":"9f098845-af24-43a7-ae3c-b815efba60cb","Type":"ContainerStarted","Data":"5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6"}
Jul 19 15:24:20.806740 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:20.806301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" event={"ID":"9f098845-af24-43a7-ae3c-b815efba60cb","Type":"ContainerStarted","Data":"ab55be6559c635d6f81ec38ff73bacdc0fc9c47d59f50c89b51cf2bba2f0dbbc"}
Jul 19 15:24:20.821342 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:20.821293    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" podStartSLOduration=2.432517013 podStartE2EDuration="2.821275245s" totalImagesPullingTime="388.758232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:24:20.82114303 +0000 UTC m=+2241.846152107" watchObservedRunningTime="2026-07-19 15:24:20.821275245 +0000 UTC m=+2241.846284321"
Jul 19 15:24:21.811190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:21.811102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:23.279612 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:23.279576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:27.830936 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:27.830901    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:24:33.794024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.793987    2577 patch_prober.go:28] interesting pod/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 19 15:24:33.794436 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.794038    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 19 15:24:33.890871 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.890842    2577 generic.go:356] "Generic (PLEG): container finished" podID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerID="076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3" exitCode=0
Jul 19 15:24:33.891040 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.890912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" event={"ID":"2171d49a-7b04-4a81-81b0-84905bfd0ab6","Type":"ContainerDied","Data":"076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"}
Jul 19 15:24:33.891306 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.891292    2577 scope.go:117] "RemoveContainer" containerID="076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"
Jul 19 15:24:33.892322 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.892295    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:24:33.892424 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.892372    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:24:33.892485 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:33.892420    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:24:34.898013 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:34.897981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" event={"ID":"2171d49a-7b04-4a81-81b0-84905bfd0ab6","Type":"ContainerStarted","Data":"aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125"}
Jul 19 15:24:34.933993 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:34.933952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:34.934380 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:34.934350    2577 patch_prober.go:28] interesting pod/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 19 15:24:34.934485 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:34.934408    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 19 15:24:35.934092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:35.934046    2577 patch_prober.go:28] interesting pod/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 19 15:24:35.934634 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:35.934113    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 19 15:24:36.934475 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:36.934059    2577 patch_prober.go:28] interesting pod/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 19 15:24:36.934475 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:36.934118    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 19 15:24:37.724548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:37.724038    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:37.933335 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:37.933290    2577 patch_prober.go:28] interesting pod/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 19 15:24:37.933559 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:37.933359    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 19 15:24:38.937630 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:38.937605    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:38.937997 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:38.937917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:38.938626 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:38.938609    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:24:38.943681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:24:38.943654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:17.311268 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.311228    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"]
Jul 19 15:25:17.314772 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.314752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:17.317341 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.317319    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:25:17.323426 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.323405    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"]
Jul 19 15:25:17.405422 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.405390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0baaf0-2ff4-43a8-b462-19cff611e22c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz\" (UID: \"8f0baaf0-2ff4-43a8-b462-19cff611e22c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:17.506588 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.506553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0baaf0-2ff4-43a8-b462-19cff611e22c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz\" (UID: \"8f0baaf0-2ff4-43a8-b462-19cff611e22c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:17.507313 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.507283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0baaf0-2ff4-43a8-b462-19cff611e22c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz\" (UID: \"8f0baaf0-2ff4-43a8-b462-19cff611e22c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:17.626712 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.626633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:17.778442 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:17.778414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"]
Jul 19 15:25:17.781244 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:25:17.781208    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0baaf0_2ff4_43a8_b462_19cff611e22c.slice/crio-018c2df058b248f158c94b7c17468daa4ea6c0b1c07f0eeab13fefbd049ebabb WatchSource:0}: Error finding container 018c2df058b248f158c94b7c17468daa4ea6c0b1c07f0eeab13fefbd049ebabb: Status 404 returned error can't find the container with id 018c2df058b248f158c94b7c17468daa4ea6c0b1c07f0eeab13fefbd049ebabb
Jul 19 15:25:18.124472 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:18.124439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz" event={"ID":"8f0baaf0-2ff4-43a8-b462-19cff611e22c","Type":"ContainerStarted","Data":"018c2df058b248f158c94b7c17468daa4ea6c0b1c07f0eeab13fefbd049ebabb"}
Jul 19 15:25:19.130195 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:19.130161    2577 generic.go:356] "Generic (PLEG): container finished" podID="8f0baaf0-2ff4-43a8-b462-19cff611e22c" containerID="63656f4b3363dd07e5cba86371d4b506829dfb3618590f23cd386020a1ba0b28" exitCode=0
Jul 19 15:25:19.130608 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:19.130251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz" event={"ID":"8f0baaf0-2ff4-43a8-b462-19cff611e22c","Type":"ContainerDied","Data":"63656f4b3363dd07e5cba86371d4b506829dfb3618590f23cd386020a1ba0b28"}
Jul 19 15:25:20.269411 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:20.269386    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:20.332721 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:20.332688    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0baaf0-2ff4-43a8-b462-19cff611e22c-maas-api-service-ca\") pod \"8f0baaf0-2ff4-43a8-b462-19cff611e22c\" (UID: \"8f0baaf0-2ff4-43a8-b462-19cff611e22c\") "
Jul 19 15:25:20.333047 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:20.333024    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f0baaf0-2ff4-43a8-b462-19cff611e22c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f0baaf0-2ff4-43a8-b462-19cff611e22c" (UID: "8f0baaf0-2ff4-43a8-b462-19cff611e22c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:25:20.434091 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:20.434007    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0baaf0-2ff4-43a8-b462-19cff611e22c-maas-api-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:25:21.141078 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:21.141051    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"
Jul 19 15:25:21.141078 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:21.141068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz" event={"ID":"8f0baaf0-2ff4-43a8-b462-19cff611e22c","Type":"ContainerDied","Data":"018c2df058b248f158c94b7c17468daa4ea6c0b1c07f0eeab13fefbd049ebabb"}
Jul 19 15:25:21.141291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:21.141112    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="018c2df058b248f158c94b7c17468daa4ea6c0b1c07f0eeab13fefbd049ebabb"
Jul 19 15:25:32.686704 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:32.686665    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"]
Jul 19 15:25:32.687103 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:32.686879    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" podUID="9f098845-af24-43a7-ae3c-b815efba60cb" containerName="maas-api" containerID="cri-o://5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6" gracePeriod=30
Jul 19 15:25:32.708450 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:32.708426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:32.817200 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:32.817144    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" podUID="9f098845-af24-43a7-ae3c-b815efba60cb" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.67:8443/health\": dial tcp 10.134.0.67:8443: connect: connection refused"
Jul 19 15:25:32.943558 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:32.943455    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:25:33.042393 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.042357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9m8t\" (UniqueName: \"kubernetes.io/projected/9f098845-af24-43a7-ae3c-b815efba60cb-kube-api-access-t9m8t\") pod \"9f098845-af24-43a7-ae3c-b815efba60cb\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") "
Jul 19 15:25:33.042633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.042423    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls\") pod \"9f098845-af24-43a7-ae3c-b815efba60cb\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") "
Jul 19 15:25:33.042633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.042534    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f098845-af24-43a7-ae3c-b815efba60cb-openshift-service-ca\") pod \"9f098845-af24-43a7-ae3c-b815efba60cb\" (UID: \"9f098845-af24-43a7-ae3c-b815efba60cb\") "
Jul 19 15:25:33.042902 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.042878    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f098845-af24-43a7-ae3c-b815efba60cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f098845-af24-43a7-ae3c-b815efba60cb" (UID: "9f098845-af24-43a7-ae3c-b815efba60cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:25:33.044440 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.044417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f098845-af24-43a7-ae3c-b815efba60cb-kube-api-access-t9m8t" (OuterVolumeSpecName: "kube-api-access-t9m8t") pod "9f098845-af24-43a7-ae3c-b815efba60cb" (UID: "9f098845-af24-43a7-ae3c-b815efba60cb"). InnerVolumeSpecName "kube-api-access-t9m8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:25:33.044648 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.044628    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f098845-af24-43a7-ae3c-b815efba60cb" (UID: "9f098845-af24-43a7-ae3c-b815efba60cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:25:33.143538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.143478    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f098845-af24-43a7-ae3c-b815efba60cb-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:25:33.143538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.143532    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f098845-af24-43a7-ae3c-b815efba60cb-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:25:33.143538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.143542    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9m8t\" (UniqueName: \"kubernetes.io/projected/9f098845-af24-43a7-ae3c-b815efba60cb-kube-api-access-t9m8t\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:25:33.204733 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.204640    2577 generic.go:356] "Generic (PLEG): container finished" podID="9f098845-af24-43a7-ae3c-b815efba60cb" containerID="5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6" exitCode=0
Jul 19 15:25:33.204733 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.204689    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"
Jul 19 15:25:33.204963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.204725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" event={"ID":"9f098845-af24-43a7-ae3c-b815efba60cb","Type":"ContainerDied","Data":"5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6"}
Jul 19 15:25:33.204963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.204779    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx" event={"ID":"9f098845-af24-43a7-ae3c-b815efba60cb","Type":"ContainerDied","Data":"ab55be6559c635d6f81ec38ff73bacdc0fc9c47d59f50c89b51cf2bba2f0dbbc"}
Jul 19 15:25:33.204963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.204806    2577 scope.go:117] "RemoveContainer" containerID="5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6"
Jul 19 15:25:33.213800 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.213780    2577 scope.go:117] "RemoveContainer" containerID="5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6"
Jul 19 15:25:33.214083 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:25:33.214062    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6\": container with ID starting with 5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6 not found: ID does not exist" containerID="5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6"
Jul 19 15:25:33.214157 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.214095    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6"} err="failed to get container status \"5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6\": rpc error: code = NotFound desc = could not find container \"5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6\": container with ID starting with 5a2f9746dd7794b64298d219839071bbe7e8ea457010b447e11ecde5c711b6f6 not found: ID does not exist"
Jul 19 15:25:33.226513 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.226472    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"]
Jul 19 15:25:33.229306 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.229283    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28015a4b-5ff855659c-7jcpx"]
Jul 19 15:25:33.539729 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:33.539700    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f098845-af24-43a7-ae3c-b815efba60cb" path="/var/lib/kubelet/pods/9f098845-af24-43a7-ae3c-b815efba60cb/volumes"
Jul 19 15:25:37.376361 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:37.376321    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"]
Jul 19 15:25:37.378891 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:37.378867    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28015a4b-2d6wz"]
Jul 19 15:25:37.379228 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:37.379209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:37.539270 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:37.539234    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0baaf0-2ff4-43a8-b462-19cff611e22c" path="/var/lib/kubelet/pods/8f0baaf0-2ff4-43a8-b462-19cff611e22c/volumes"
Jul 19 15:25:39.671167 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:39.671131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:44.813725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:44.813694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:59.686211 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:59.686136    2577 scope.go:117] "RemoveContainer" containerID="c179d41efa41bf56eb6313a8627aab1e3cdd1dc91f58cee772ab25277d8dee27"
Jul 19 15:25:59.694318 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:25:59.694292    2577 scope.go:117] "RemoveContainer" containerID="282bd042f623e9a811e78054e8569ad369442a0b80c3e40a0809c255514d8e6d"
Jul 19 15:26:04.857533 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:04.857476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:04.958412 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:04.958368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:20.949986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:20.949958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:21.715082 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:21.715039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:27.154232 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:27.154198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:27.213775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:27.213744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:27.420459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:27.420375    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"]
Jul 19 15:26:27.420650 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:27.420602    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy" containerID="cri-o://aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125" gracePeriod=30
Jul 19 15:26:27.450681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:27.450635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:27.695234 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:27.695131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:28.786896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:28.786848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:28.884420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:28.884365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:28.987232 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:28.987195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:29.784883 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:29.784844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:29.883165 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:29.883133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:29.982881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:29.982851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:31.420177 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:31.420148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:32.688508 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.688435    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.755784    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-token\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.755858    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkr7z\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-kube-api-access-xkr7z\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.755903    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-certs\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.755930    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-data\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.755962    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istiod-ca-cert\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.755988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-credential-socket\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.756036    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-socket\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.756063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-podinfo\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.756516 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.756092    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-envoy\") pod \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\" (UID: \"2171d49a-7b04-4a81-81b0-84905bfd0ab6\") "
Jul 19 15:26:32.758615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.757093    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:26:32.758615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.758057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-data" (OuterVolumeSpecName: "istio-data") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:26:32.758615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.758116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:26:32.758615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.758247    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:26:32.758615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.758260    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:26:32.770022 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.769804    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-kube-api-access-xkr7z" (OuterVolumeSpecName: "kube-api-access-xkr7z") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "kube-api-access-xkr7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:26:32.771256 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.771216    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-token" (OuterVolumeSpecName: "istio-token") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:26:32.771387 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.771303    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:26:32.771932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.771900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2171d49a-7b04-4a81-81b0-84905bfd0ab6" (UID: "2171d49a-7b04-4a81-81b0-84905bfd0ab6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:26:32.856967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.856930    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkr7z\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-kube-api-access-xkr7z\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.856967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.856959    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.856967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.856969    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.857196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.856977    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.857196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.856986    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.857196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.856994    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.857196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.857002    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.857196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.857010    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:32.857196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:32.857018    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2171d49a-7b04-4a81-81b0-84905bfd0ab6-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:26:33.504696 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.504668    2577 generic.go:356] "Generic (PLEG): container finished" podID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerID="aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125" exitCode=0
Jul 19 15:26:33.504873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.504711    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"
Jul 19 15:26:33.504873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.504750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" event={"ID":"2171d49a-7b04-4a81-81b0-84905bfd0ab6","Type":"ContainerDied","Data":"aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125"}
Jul 19 15:26:33.504873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.504803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc" event={"ID":"2171d49a-7b04-4a81-81b0-84905bfd0ab6","Type":"ContainerDied","Data":"e783113ebce2e452f1d8b7ae986782c9b90271b825bc986229094718b56c6b2d"}
Jul 19 15:26:33.504873 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.504824    2577 scope.go:117] "RemoveContainer" containerID="aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125"
Jul 19 15:26:33.513910 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.513894    2577 scope.go:117] "RemoveContainer" containerID="076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"
Jul 19 15:26:33.521596 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.521580    2577 scope.go:117] "RemoveContainer" containerID="aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125"
Jul 19 15:26:33.521848 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:26:33.521826    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125\": container with ID starting with aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125 not found: ID does not exist" containerID="aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125"
Jul 19 15:26:33.521909 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.521856    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125"} err="failed to get container status \"aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125\": rpc error: code = NotFound desc = could not find container \"aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125\": container with ID starting with aa976d69a852d1316c8c6bce6f64dce85a154754ab80a6cc4f4ef9348ffc0125 not found: ID does not exist"
Jul 19 15:26:33.521909 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.521873    2577 scope.go:117] "RemoveContainer" containerID="076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"
Jul 19 15:26:33.522132 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:26:33.522109    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3\": container with ID starting with 076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3 not found: ID does not exist" containerID="076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"
Jul 19 15:26:33.522172 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.522149    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3"} err="failed to get container status \"076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3\": rpc error: code = NotFound desc = could not find container \"076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3\": container with ID starting with 076a0f6b93ddeec71e347fb68368898edba895ca8d8aa79b471ca6bb178ee6d3 not found: ID does not exist"
Jul 19 15:26:33.524922 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.524901    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"]
Jul 19 15:26:33.528608 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.528588    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-28015a4b-openshift-default-6867f46479-djrpc"]
Jul 19 15:26:33.539872 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:33.539854    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" path="/var/lib/kubelet/pods/2171d49a-7b04-4a81-81b0-84905bfd0ab6/volumes"
Jul 19 15:26:37.169760 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:37.169707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:37.273846 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:37.273821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:37.871123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:37.871088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:43.939026 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:43.938975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:44.047007 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:44.046961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:44.139051 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:44.139012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:54.683744 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:54.683701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:55.051465 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:55.051426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:59.718635 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:26:59.718607    2577 scope.go:117] "RemoveContainer" containerID="816b0089b2f0a466c138067f87f362840238c633327e218dc34377e6475129fe"
Jul 19 15:27:11.973199 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:11.973140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:12.024836 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:12.024805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:16.017415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:16.017380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:22.007092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.007007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:22.107594 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.107558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:22.380249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.380160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:22.673208 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.673120    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"]
Jul 19 15:27:22.674029 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674002    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy"
Jul 19 15:27:22.674029 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674027    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674047    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f098845-af24-43a7-ae3c-b815efba60cb" containerName="maas-api"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674056    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f098845-af24-43a7-ae3c-b815efba60cb" containerName="maas-api"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674091    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0baaf0-2ff4-43a8-b462-19cff611e22c" containerName="revoke-keys"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674101    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0baaf0-2ff4-43a8-b462-19cff611e22c" containerName="revoke-keys"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674180    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f098845-af24-43a7-ae3c-b815efba60cb" containerName="maas-api"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674190    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674203    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0baaf0-2ff4-43a8-b462-19cff611e22c" containerName="revoke-keys"
Jul 19 15:27:22.674240 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.674214    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy"
Jul 19 15:27:22.677420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.677397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.680318 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.680293    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-9d5834-openshift-default-dockercfg-fhgbl\""
Jul 19 15:27:22.689834 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.689813    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"]
Jul 19 15:27:22.703625 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.703605    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:22.774666 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-certs\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774666 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-credential-socket\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f05437-d1d9-4615-8357-d67cac1c8b75-istiod-ca-cert\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-envoy\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-podinfo\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-token\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rd5w\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-kube-api-access-8rd5w\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-data\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.774881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.774879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-socket\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.875625 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-envoy\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.875817 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-podinfo\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.875817 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-token\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.875817 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rd5w\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-kube-api-access-8rd5w\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.875817 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-data\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.875817 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-socket\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876095 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-certs\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876095 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-credential-socket\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876095 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.875946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f05437-d1d9-4615-8357-d67cac1c8b75-istiod-ca-cert\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876268 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.876214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-socket\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876389 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.876355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-data\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876624 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.876600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-credential-socket\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876717 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.876656    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f05437-d1d9-4615-8357-d67cac1c8b75-istiod-ca-cert\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.876788 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.876772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-certs\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.877983 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.877960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-envoy\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.878108 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.878091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-podinfo\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.884422 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.884394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-token\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.884556 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.884539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rd5w\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-kube-api-access-8rd5w\") pod \"e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") " pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:22.990030 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:22.990002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:23.049360 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.049329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:23.127104 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.127072    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"]
Jul 19 15:27:23.128474 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:27:23.128447    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f05437_d1d9_4615_8357_d67cac1c8b75.slice/crio-f3abe76258fbcbf88786e6946b86c88c1e4bf68bea0643e49dc1686461da34b0 WatchSource:0}: Error finding container f3abe76258fbcbf88786e6946b86c88c1e4bf68bea0643e49dc1686461da34b0: Status 404 returned error can't find the container with id f3abe76258fbcbf88786e6946b86c88c1e4bf68bea0643e49dc1686461da34b0
Jul 19 15:27:23.130712 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.130670    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:23.130811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.130754    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:23.130811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.130796    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:23.746345 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.746311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" event={"ID":"f4f05437-d1d9-4615-8357-d67cac1c8b75","Type":"ContainerStarted","Data":"452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"}
Jul 19 15:27:23.746564 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.746361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" event={"ID":"f4f05437-d1d9-4615-8357-d67cac1c8b75","Type":"ContainerStarted","Data":"f3abe76258fbcbf88786e6946b86c88c1e4bf68bea0643e49dc1686461da34b0"}
Jul 19 15:27:23.767031 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.766981    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podStartSLOduration=1.766940994 podStartE2EDuration="1.766940994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:27:23.765725801 +0000 UTC m=+2424.790734888" watchObservedRunningTime="2026-07-19 15:27:23.766940994 +0000 UTC m=+2424.791950128"
Jul 19 15:27:23.990124 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:23.990094    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:24.995968 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:24.995938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:25.722547 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.722517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:25.756380 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.756352    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:25.757353 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.757337    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:25.783093 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.783064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:25.968855 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.968824    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"]
Jul 19 15:27:25.969215 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.969202    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy"
Jul 19 15:27:25.969262 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.969217    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2171d49a-7b04-4a81-81b0-84905bfd0ab6" containerName="istio-proxy"
Jul 19 15:27:25.972529 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.972509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:25.975004 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.974982    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9d5834-serving-cert\""
Jul 19 15:27:25.983030 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:25.983006    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"]
Jul 19 15:27:26.002183 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.002160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:26.103037 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.103008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.103190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.103126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvbgc\" (UniqueName: \"kubernetes.io/projected/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-kube-api-access-lvbgc\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.103190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.103160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-maas-api-tls\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.108265 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.108238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:26.204022 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.203988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.204627 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.204059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvbgc\" (UniqueName: \"kubernetes.io/projected/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-kube-api-access-lvbgc\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.204627 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.204096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-maas-api-tls\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.204871 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.204849    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.206567 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.206543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-maas-api-tls\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.212257 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.212231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvbgc\" (UniqueName: \"kubernetes.io/projected/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-kube-api-access-lvbgc\") pod \"maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.284918 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.284853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:26.420203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.420172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"]
Jul 19 15:27:26.421778 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:27:26.421753    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8587e9a7_d0b6_4ae9_b16d_6bd86d5a4046.slice/crio-488645193a6db18b4f36fea488ea4ff1a83cd10c6a8580dbf68961aff319cdcc WatchSource:0}: Error finding container 488645193a6db18b4f36fea488ea4ff1a83cd10c6a8580dbf68961aff319cdcc: Status 404 returned error can't find the container with id 488645193a6db18b4f36fea488ea4ff1a83cd10c6a8580dbf68961aff319cdcc
Jul 19 15:27:26.423169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.423148    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:27:26.465034 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.465005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:26.570001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.569936    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:26.762356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:26.762325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p" event={"ID":"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046","Type":"ContainerStarted","Data":"488645193a6db18b4f36fea488ea4ff1a83cd10c6a8580dbf68961aff319cdcc"}
Jul 19 15:27:27.768830 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:27.768789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p" event={"ID":"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046","Type":"ContainerStarted","Data":"389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09"}
Jul 19 15:27:27.786420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:27.786375    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p" podStartSLOduration=2.318253577 podStartE2EDuration="2.786361055s" totalImagesPullingTime="468.107478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:27:27.784512963 +0000 UTC m=+2428.809522032" watchObservedRunningTime="2026-07-19 15:27:27.786361055 +0000 UTC m=+2428.811370130"
Jul 19 15:27:28.774283 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:28.774251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:30.252557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.252519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:30.597806 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.597721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:30.881407 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.881324    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"]
Jul 19 15:27:30.885468 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.885445    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:30.888116 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.888095    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-42a367-openshift-default-dockercfg-9jsp6\""
Jul 19 15:27:30.896396 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.896369    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"]
Jul 19 15:27:30.911944 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:30.911918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:31.048249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cbgq\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-kube-api-access-7cbgq\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048429 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-data\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048429 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-envoy\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048555 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-credential-socket\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048555 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istiod-ca-cert\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048555 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-token\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048766 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-podinfo\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048766 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-socket\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.048766 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.048694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-certs\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149130 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cbgq\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-kube-api-access-7cbgq\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149130 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-data\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149333 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149160    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-envoy\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149333 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-credential-socket\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istiod-ca-cert\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-token\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-podinfo\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-socket\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-certs\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-data\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.149773 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-credential-socket\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.150001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-certs\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.150115 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.149999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-socket\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.150198 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.150184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istiod-ca-cert\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.151932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.151906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-envoy\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.152232 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.152211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-podinfo\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.156934 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.156908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-token\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.157055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.157038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cbgq\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-kube-api-access-7cbgq\") pod \"e2e-inf-b-42a367-openshift-default-55755f8989-q58zf\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") " pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.199059 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.199032    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:31.258180 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.258145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:31.346693 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.346665    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"]
Jul 19 15:27:31.348256 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:27:31.348230    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8872a13b_d56a_40d2_9b44_9153a1a64ad5.slice/crio-9f74f7c2e760c5677204f226a8c245bb9630fb352cf27b9dcd595bc5d288ecb8 WatchSource:0}: Error finding container 9f74f7c2e760c5677204f226a8c245bb9630fb352cf27b9dcd595bc5d288ecb8: Status 404 returned error can't find the container with id 9f74f7c2e760c5677204f226a8c245bb9630fb352cf27b9dcd595bc5d288ecb8
Jul 19 15:27:31.350586 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.350555    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:31.350657 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.350621    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:31.350657 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.350651    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:31.790021 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.789991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" event={"ID":"8872a13b-d56a-40d2-9b44-9153a1a64ad5","Type":"ContainerStarted","Data":"d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"}
Jul 19 15:27:31.790202 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.790040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" event={"ID":"8872a13b-d56a-40d2-9b44-9153a1a64ad5","Type":"ContainerStarted","Data":"9f74f7c2e760c5677204f226a8c245bb9630fb352cf27b9dcd595bc5d288ecb8"}
Jul 19 15:27:31.810080 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:31.810035    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podStartSLOduration=1.8100220949999999 podStartE2EDuration="1.810022095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:27:31.808166084 +0000 UTC m=+2432.833175182" watchObservedRunningTime="2026-07-19 15:27:31.810022095 +0000 UTC m=+2432.835031169"
Jul 19 15:27:32.199791 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:32.199708    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:32.204518 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:32.204468    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:32.794808 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:32.794779    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:32.795741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:32.795718    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:33.744642 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.744611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:33.840521 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.839036    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"]
Jul 19 15:27:33.850554 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.850012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:33.853020 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.852999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:33.855754 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.855725    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-42a367-serving-cert\""
Jul 19 15:27:33.861546 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.861522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"]
Jul 19 15:27:33.934513 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.932031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:33.975993 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.975957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlmmt\" (UniqueName: \"kubernetes.io/projected/c680bbc9-1c55-4460-a8e2-7404bb729d0c-kube-api-access-xlmmt\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:33.976169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.976067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:33.976169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:33.976109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c680bbc9-1c55-4460-a8e2-7404bb729d0c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.077122 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.077042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlmmt\" (UniqueName: \"kubernetes.io/projected/c680bbc9-1c55-4460-a8e2-7404bb729d0c-kube-api-access-xlmmt\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.077122 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.077095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.077304 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:27:34.077180    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-serving-cert: secret "maas-api-e2e-inf-b-42a367-serving-cert" not found
Jul 19 15:27:34.077304 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:27:34.077228    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls podName:c680bbc9-1c55-4460-a8e2-7404bb729d0c nodeName:}" failed. No retries permitted until 2026-07-19 15:27:34.5772128 +0000 UTC m=+2435.602221853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls") pod "maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" (UID: "c680bbc9-1c55-4460-a8e2-7404bb729d0c") : secret "maas-api-e2e-inf-b-42a367-serving-cert" not found
Jul 19 15:27:34.077304 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.077260    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c680bbc9-1c55-4460-a8e2-7404bb729d0c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.077924 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.077907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c680bbc9-1c55-4460-a8e2-7404bb729d0c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.086043 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.086018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlmmt\" (UniqueName: \"kubernetes.io/projected/c680bbc9-1c55-4460-a8e2-7404bb729d0c-kube-api-access-xlmmt\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.348471 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.348371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:34.449641 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.449606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:34.550053 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.550020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:34.583276 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.583240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.585838 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.585812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls\") pod \"maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.767549 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.767515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:34.786230 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.786206    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:27:34.963597 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:34.963564    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"]
Jul 19 15:27:34.965016 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:27:34.964985    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc680bbc9_1c55_4460_a8e2_7404bb729d0c.slice/crio-d3db42c0d7666426e93c139b9502c35fdf0b4338be712a4de419cbc62387ec8c WatchSource:0}: Error finding container d3db42c0d7666426e93c139b9502c35fdf0b4338be712a4de419cbc62387ec8c: Status 404 returned error can't find the container with id d3db42c0d7666426e93c139b9502c35fdf0b4338be712a4de419cbc62387ec8c
Jul 19 15:27:35.813237 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:35.813156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" event={"ID":"c680bbc9-1c55-4460-a8e2-7404bb729d0c","Type":"ContainerStarted","Data":"13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490"}
Jul 19 15:27:35.813237 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:35.813208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" event={"ID":"c680bbc9-1c55-4460-a8e2-7404bb729d0c","Type":"ContainerStarted","Data":"d3db42c0d7666426e93c139b9502c35fdf0b4338be712a4de419cbc62387ec8c"}
Jul 19 15:27:35.813438 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:35.813264    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:35.831554 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:35.831483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" podStartSLOduration=2.266049434 podStartE2EDuration="2.831464987s" totalImagesPullingTime="565.415553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:27:35.828988885 +0000 UTC m=+2436.853997963" watchObservedRunningTime="2026-07-19 15:27:35.831464987 +0000 UTC m=+2436.856474067"
Jul 19 15:27:38.272253 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:38.272222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:39.837809 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:39.837776    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerID="452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa" exitCode=0
Jul 19 15:27:39.838254 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:39.837852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" event={"ID":"f4f05437-d1d9-4615-8357-d67cac1c8b75","Type":"ContainerDied","Data":"452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"}
Jul 19 15:27:39.838254 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:39.838232    2577 scope.go:117] "RemoveContainer" containerID="452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"
Jul 19 15:27:39.839251 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:39.839221    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:39.839362 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:39.839307    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:39.839362 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:39.839346    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:40.756572 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:40.756529    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:40.848347 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:40.848314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" event={"ID":"f4f05437-d1d9-4615-8357-d67cac1c8b75","Type":"ContainerStarted","Data":"9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f"}
Jul 19 15:27:40.990648 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:40.990607    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:40.991899 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:40.991873    2577 patch_prober.go:28] interesting pod/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 19 15:27:40.992019 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:40.991923    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 19 15:27:41.836099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:41.836059    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:27:41.991216 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:41.991187    2577 patch_prober.go:28] interesting pod/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 19 15:27:41.991670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:41.991593    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 19 15:27:42.991081 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:42.991036    2577 patch_prober.go:28] interesting pod/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 19 15:27:42.991338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:42.991098    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 19 15:27:43.991314 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:43.991266    2577 patch_prober.go:28] interesting pod/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 19 15:27:43.991515 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:43.991333    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 19 15:27:44.990926 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:44.990880    2577 patch_prober.go:28] interesting pod/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 19 15:27:44.991133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:44.990942    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 19 15:27:45.994681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:45.994647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:45.995179 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:45.994996    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:45.995630 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:45.995609    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:27:47.198360 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:47.198324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:47.296083 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:47.296048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:47.968792 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:47.968334    2577 patch_prober.go:28] interesting pod/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": read tcp 10.134.0.2:41550->10.134.0.71:15021: read: connection reset by peer" start-of-body=
Jul 19 15:27:47.968792 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:47.968408    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": read tcp 10.134.0.2:41550->10.134.0.71:15021: read: connection reset by peer"
Jul 19 15:27:48.895735 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:48.895700    2577 generic.go:356] "Generic (PLEG): container finished" podID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerID="d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c" exitCode=0
Jul 19 15:27:48.896145 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:48.895774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" event={"ID":"8872a13b-d56a-40d2-9b44-9153a1a64ad5","Type":"ContainerDied","Data":"d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"}
Jul 19 15:27:48.896145 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:48.896138    2577 scope.go:117] "RemoveContainer" containerID="d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"
Jul 19 15:27:48.897093 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:48.897067    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:48.897190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:48.897129    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:48.897190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:48.897163    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:27:49.902926 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:49.902892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" event={"ID":"8872a13b-d56a-40d2-9b44-9153a1a64ad5","Type":"ContainerStarted","Data":"f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37"}
Jul 19 15:27:50.200085 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:50.200003    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:50.200262 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:50.200207    2577 patch_prober.go:28] interesting pod/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 19 15:27:50.200338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:50.200264    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 19 15:27:51.199881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:51.199844    2577 patch_prober.go:28] interesting pod/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 19 15:27:51.200289 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:51.199899    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 19 15:27:52.200382 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:52.200346    2577 patch_prober.go:28] interesting pod/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 19 15:27:52.200802 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:52.200401    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 19 15:27:53.199866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:53.199828    2577 patch_prober.go:28] interesting pod/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 19 15:27:53.200058 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:53.199884    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 19 15:27:54.203959 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:54.203931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:54.204420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:54.204200    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:54.204838 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:54.204821    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:27:55.307260 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:55.307218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:55.406420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:55.406391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:27:59.741951 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:27:59.741915    2577 scope.go:117] "RemoveContainer" containerID="cbaa5e945b2b004c65476217eb3f5f60ee6e3641f2076b29cb57a5766d2046d2"
Jul 19 15:28:07.188318 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:07.188284    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:28:31.675409 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:31.675376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:28:50.648037 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:50.647979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:28:52.380294 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:52.380182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:28:53.374858 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:53.374806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:28:53.960083 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:53.960049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:28:54.061458 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:28:54.061427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:03.899039 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:03.899005    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"]
Jul 19 15:29:03.899539 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:03.899216    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p" podUID="8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" containerName="maas-api" containerID="cri-o://389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09" gracePeriod=30
Jul 19 15:29:03.962542 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:03.962511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:04.151024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.150968    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:29:04.286838 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.286806    2577 generic.go:356] "Generic (PLEG): container finished" podID="8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" containerID="389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09" exitCode=0
Jul 19 15:29:04.287014 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.286866    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"
Jul 19 15:29:04.287014 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.286898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p" event={"ID":"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046","Type":"ContainerDied","Data":"389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09"}
Jul 19 15:29:04.287014 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.286980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p" event={"ID":"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046","Type":"ContainerDied","Data":"488645193a6db18b4f36fea488ea4ff1a83cd10c6a8580dbf68961aff319cdcc"}
Jul 19 15:29:04.287014 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.287005    2577 scope.go:117] "RemoveContainer" containerID="389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09"
Jul 19 15:29:04.288050 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.288034    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvbgc\" (UniqueName: \"kubernetes.io/projected/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-kube-api-access-lvbgc\") pod \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") "
Jul 19 15:29:04.288145 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.288130    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-maas-api-tls\") pod \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") "
Jul 19 15:29:04.288183 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.288169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-openshift-service-ca\") pod \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\" (UID: \"8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046\") "
Jul 19 15:29:04.288596 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.288570    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" (UID: "8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:29:04.290193 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.290167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" (UID: "8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:29:04.290350 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.290329    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-kube-api-access-lvbgc" (OuterVolumeSpecName: "kube-api-access-lvbgc") pod "8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" (UID: "8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046"). InnerVolumeSpecName "kube-api-access-lvbgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:04.306643 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.306623    2577 scope.go:117] "RemoveContainer" containerID="389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09"
Jul 19 15:29:04.306917 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:04.306897    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09\": container with ID starting with 389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09 not found: ID does not exist" containerID="389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09"
Jul 19 15:29:04.307006 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.306941    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09"} err="failed to get container status \"389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09\": rpc error: code = NotFound desc = could not find container \"389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09\": container with ID starting with 389d956d46ef06cfc3a5e0bf495d451d0322e8782c15635fa0be450e1cd8de09 not found: ID does not exist"
Jul 19 15:29:04.389686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.389664    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvbgc\" (UniqueName: \"kubernetes.io/projected/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-kube-api-access-lvbgc\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:04.389686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.389685    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:04.389811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.389694    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:04.607528 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.607489    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"]
Jul 19 15:29:04.611685 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:04.611664    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9d5834-5d69455bcf-2dp5p"]
Jul 19 15:29:05.539642 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:05.539609    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" path="/var/lib/kubelet/pods/8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046/volumes"
Jul 19 15:29:08.568658 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:08.568621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:08.976623 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:08.976596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:14.721733 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:14.721700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:14.792676 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:14.792639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:14.973196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:14.973124    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"]
Jul 19 15:29:14.973392 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:14.973369    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" containerID="cri-o://9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f" gracePeriod=30
Jul 19 15:29:14.992705 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:14.992678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:15.228138 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:15.228100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:15.825164 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:15.825133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:15.925703 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:15.925675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:15.997819 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:15.997773    2577 patch_prober.go:28] interesting pod/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:29:15.998003 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:15.997855    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:29:20.235263 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.235201    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:29:20.333941 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.333913    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-envoy\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.333968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-podinfo\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.333988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-certs\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334040    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-credential-socket\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334064    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-data\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334101    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-token\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rd5w\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-kube-api-access-8rd5w\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334170    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-socket\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f05437-d1d9-4615-8357-d67cac1c8b75-istiod-ca-cert\") pod \"f4f05437-d1d9-4615-8357-d67cac1c8b75\" (UID: \"f4f05437-d1d9-4615-8357-d67cac1c8b75\") "
Jul 19 15:29:20.334542 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334348    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:20.334542 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334364    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:20.334673 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334579    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.334673 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334585    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:20.334673 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334599    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.334847 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334698    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f05437-d1d9-4615-8357-d67cac1c8b75-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:29:20.334908 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.334861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:20.336224 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.336204    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:29:20.336393 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.336375    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-kube-api-access-8rd5w" (OuterVolumeSpecName: "kube-api-access-8rd5w") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "kube-api-access-8rd5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:20.336546 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.336527    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:20.336611 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.336594    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4f05437-d1d9-4615-8357-d67cac1c8b75" (UID: "f4f05437-d1d9-4615-8357-d67cac1c8b75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:20.371297 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.371270    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerID="9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f" exitCode=0
Jul 19 15:29:20.371408 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.371324    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"
Jul 19 15:29:20.371408 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.371325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" event={"ID":"f4f05437-d1d9-4615-8357-d67cac1c8b75","Type":"ContainerDied","Data":"9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f"}
Jul 19 15:29:20.371543 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.371448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w" event={"ID":"f4f05437-d1d9-4615-8357-d67cac1c8b75","Type":"ContainerDied","Data":"f3abe76258fbcbf88786e6946b86c88c1e4bf68bea0643e49dc1686461da34b0"}
Jul 19 15:29:20.371543 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.371478    2577 scope.go:117] "RemoveContainer" containerID="9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f"
Jul 19 15:29:20.380380 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.380363    2577 scope.go:117] "RemoveContainer" containerID="452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"
Jul 19 15:29:20.388452 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.388434    2577 scope.go:117] "RemoveContainer" containerID="9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f"
Jul 19 15:29:20.388775 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:20.388748    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f\": container with ID starting with 9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f not found: ID does not exist" containerID="9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f"
Jul 19 15:29:20.388888 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.388774    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f"} err="failed to get container status \"9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f\": rpc error: code = NotFound desc = could not find container \"9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f\": container with ID starting with 9bd85b308156bb3e08d072439c92376d80c73f945d7f763ffa3e0b54b58a453f not found: ID does not exist"
Jul 19 15:29:20.388888 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.388791    2577 scope.go:117] "RemoveContainer" containerID="452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"
Jul 19 15:29:20.389081 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:20.389052    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa\": container with ID starting with 452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa not found: ID does not exist" containerID="452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"
Jul 19 15:29:20.389190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.389090    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa"} err="failed to get container status \"452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa\": rpc error: code = NotFound desc = could not find container \"452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa\": container with ID starting with 452ee72b86040253a76bef836c724a8ba73007b77e5bc0c7873e771bb12cd7fa not found: ID does not exist"
Jul 19 15:29:20.392298 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.392275    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"]
Jul 19 15:29:20.397171 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.397151    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-9d5834-openshift-default-cb659f8dc-f4z5w"]
Jul 19 15:29:20.435734 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435708    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.435734 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435729    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4f05437-d1d9-4615-8357-d67cac1c8b75-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.435734 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435740    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.435933 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435748    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.435933 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435756    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.435933 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435765    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:20.435933 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:20.435773    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rd5w\" (UniqueName: \"kubernetes.io/projected/f4f05437-d1d9-4615-8357-d67cac1c8b75-kube-api-access-8rd5w\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:21.539875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:21.539842    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" path="/var/lib/kubelet/pods/f4f05437-d1d9-4615-8357-d67cac1c8b75/volumes"
Jul 19 15:29:25.727761 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:25.727725    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"]
Jul 19 15:29:25.728235 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:25.727973    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" podUID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" containerName="maas-api" containerID="cri-o://13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490" gracePeriod=30
Jul 19 15:29:25.748302 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:25.748272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:26.818947 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.818907    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" podUID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.72:8443/health\": dial tcp 10.134.0.72:8443: connect: connection refused"
Jul 19 15:29:26.975392 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.975368    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:29:26.993809 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.993773    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlmmt\" (UniqueName: \"kubernetes.io/projected/c680bbc9-1c55-4460-a8e2-7404bb729d0c-kube-api-access-xlmmt\") pod \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") "
Jul 19 15:29:26.993956 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.993820    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls\") pod \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") "
Jul 19 15:29:26.993956 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.993853    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c680bbc9-1c55-4460-a8e2-7404bb729d0c-openshift-service-ca\") pod \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\" (UID: \"c680bbc9-1c55-4460-a8e2-7404bb729d0c\") "
Jul 19 15:29:26.994526 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.994462    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c680bbc9-1c55-4460-a8e2-7404bb729d0c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c680bbc9-1c55-4460-a8e2-7404bb729d0c" (UID: "c680bbc9-1c55-4460-a8e2-7404bb729d0c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:29:26.996451 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.996430    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c680bbc9-1c55-4460-a8e2-7404bb729d0c-kube-api-access-xlmmt" (OuterVolumeSpecName: "kube-api-access-xlmmt") pod "c680bbc9-1c55-4460-a8e2-7404bb729d0c" (UID: "c680bbc9-1c55-4460-a8e2-7404bb729d0c"). InnerVolumeSpecName "kube-api-access-xlmmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:26.996633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:26.996567    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c680bbc9-1c55-4460-a8e2-7404bb729d0c" (UID: "c680bbc9-1c55-4460-a8e2-7404bb729d0c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:29:27.094640 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.094574    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlmmt\" (UniqueName: \"kubernetes.io/projected/c680bbc9-1c55-4460-a8e2-7404bb729d0c-kube-api-access-xlmmt\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:27.094640 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.094597    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c680bbc9-1c55-4460-a8e2-7404bb729d0c-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:27.094640 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.094608    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c680bbc9-1c55-4460-a8e2-7404bb729d0c-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:27.408910 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.408820    2577 generic.go:356] "Generic (PLEG): container finished" podID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" containerID="13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490" exitCode=0
Jul 19 15:29:27.408910 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.408867    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"
Jul 19 15:29:27.408910 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.408886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" event={"ID":"c680bbc9-1c55-4460-a8e2-7404bb729d0c","Type":"ContainerDied","Data":"13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490"}
Jul 19 15:29:27.409137 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.408936    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd" event={"ID":"c680bbc9-1c55-4460-a8e2-7404bb729d0c","Type":"ContainerDied","Data":"d3db42c0d7666426e93c139b9502c35fdf0b4338be712a4de419cbc62387ec8c"}
Jul 19 15:29:27.409137 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.408959    2577 scope.go:117] "RemoveContainer" containerID="13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490"
Jul 19 15:29:27.418742 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.418725    2577 scope.go:117] "RemoveContainer" containerID="13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490"
Jul 19 15:29:27.419005 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:27.418987    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490\": container with ID starting with 13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490 not found: ID does not exist" containerID="13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490"
Jul 19 15:29:27.419061 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.419013    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490"} err="failed to get container status \"13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490\": rpc error: code = NotFound desc = could not find container \"13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490\": container with ID starting with 13ba43cdda928756631a53ddd75bb9fd972aa85d2992178ed5ea4c36f8d2c490 not found: ID does not exist"
Jul 19 15:29:27.429016 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.428990    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"]
Jul 19 15:29:27.435243 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.435223    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-42a367-fc8f5fd66-2tpsd"]
Jul 19 15:29:27.539050 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:27.539024    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" path="/var/lib/kubelet/pods/c680bbc9-1c55-4460-a8e2-7404bb729d0c/volumes"
Jul 19 15:29:30.421896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:30.421855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:30.827698 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:30.827657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:36.719218 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:36.719179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:36.791801 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:36.791772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:36.967850 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:36.967814    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"]
Jul 19 15:29:36.968119 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:36.968064    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" containerID="cri-o://f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37" gracePeriod=30
Jul 19 15:29:36.987608 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:36.987582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:37.242636 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.242549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:37.595971 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.595881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:37.892760 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.892678    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"]
Jul 19 15:29:37.893151 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893135    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy"
Jul 19 15:29:37.893151 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893152    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy"
Jul 19 15:29:37.893250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893166    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy"
Jul 19 15:29:37.893250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893171    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy"
Jul 19 15:29:37.893250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893186    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" containerName="maas-api"
Jul 19 15:29:37.893250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" containerName="maas-api"
Jul 19 15:29:37.893250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893200    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" containerName="maas-api"
Jul 19 15:29:37.893250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893205    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" containerName="maas-api"
Jul 19 15:29:37.893451 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893266    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c680bbc9-1c55-4460-a8e2-7404bb729d0c" containerName="maas-api"
Jul 19 15:29:37.893451 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893275    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy"
Jul 19 15:29:37.893451 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893284    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f05437-d1d9-4615-8357-d67cac1c8b75" containerName="istio-proxy"
Jul 19 15:29:37.893451 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.893290    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8587e9a7-d0b6-4ae9-b16d-6bd86d5a4046" containerName="maas-api"
Jul 19 15:29:37.896525 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.896489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.899183 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.899164    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-20b742-openshift-default-dockercfg-jdvdm\""
Jul 19 15:29:37.906959 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.906929    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"]
Jul 19 15:29:37.926596 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.926561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:37.994089 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-credential-socket\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994104    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-socket\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-envoy\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10121645-966b-42a4-b65a-d800bad6b04a-istiod-ca-cert\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xbjc\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-kube-api-access-2xbjc\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-data\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10121645-966b-42a4-b65a-d800bad6b04a-istio-podinfo\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994656 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-istio-token\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:37.994656 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:37.994388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-certs\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095745 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-istio-token\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-certs\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-credential-socket\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-socket\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-envoy\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10121645-966b-42a4-b65a-d800bad6b04a-istiod-ca-cert\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xbjc\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-kube-api-access-2xbjc\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.095932 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-data\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.096302 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.095944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10121645-966b-42a4-b65a-d800bad6b04a-istio-podinfo\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.096302 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.096181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-certs\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.096635 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.096606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-socket\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.096774 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.096687    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-credential-socket\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.096774 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.096754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-data\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.096955 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.096931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10121645-966b-42a4-b65a-d800bad6b04a-istiod-ca-cert\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.098309 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.098276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-envoy\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.098471 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.098452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10121645-966b-42a4-b65a-d800bad6b04a-istio-podinfo\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.104250 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.104229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xbjc\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-kube-api-access-2xbjc\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.104694 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.104675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-istio-token\") pod \"e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") " pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.209070 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.208987    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:38.349981 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.349951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"]
Jul 19 15:29:38.352542 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:29:38.352489    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10121645_966b_42a4_b65a_d800bad6b04a.slice/crio-a06c0a4a24a79b577c6bc8985f72c611acdafcda33ad8f13921b469b2d17f738 WatchSource:0}: Error finding container a06c0a4a24a79b577c6bc8985f72c611acdafcda33ad8f13921b469b2d17f738: Status 404 returned error can't find the container with id a06c0a4a24a79b577c6bc8985f72c611acdafcda33ad8f13921b469b2d17f738
Jul 19 15:29:38.355198 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.355165    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:38.355291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.355230    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:38.355291 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.355259    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:38.465722 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.465631    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" event={"ID":"10121645-966b-42a4-b65a-d800bad6b04a","Type":"ContainerStarted","Data":"783c5865339e44f7896aa9909da44e47576221adad2e07a4deea88e8c23c3fd1"}
Jul 19 15:29:38.465722 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.465692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" event={"ID":"10121645-966b-42a4-b65a-d800bad6b04a","Type":"ContainerStarted","Data":"a06c0a4a24a79b577c6bc8985f72c611acdafcda33ad8f13921b469b2d17f738"}
Jul 19 15:29:38.486179 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.486133    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" podStartSLOduration=1.486119067 podStartE2EDuration="1.486119067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:29:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:29:38.484517068 +0000 UTC m=+2559.509526213" watchObservedRunningTime="2026-07-19 15:29:38.486119067 +0000 UTC m=+2559.511128141"
Jul 19 15:29:38.510332 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:38.510303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:39.206471 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:39.206433    2577 patch_prober.go:28] interesting pod/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:29:39.206862 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:39.206489    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:29:39.209338 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:39.209309    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:39.214372 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:39.214343    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:39.470281 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:39.470208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:39.471224 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:39.471201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:40.456819 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.456773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:40.520527 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.520287    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"]
Jul 19 15:29:40.524010 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.523990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.526405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.526386    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-20b742-serving-cert\""
Jul 19 15:29:40.535068 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.535046    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"]
Jul 19 15:29:40.619301 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.619257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glz58\" (UniqueName: \"kubernetes.io/projected/cb2a17fb-fce3-4412-b60f-28d92dceb71e-kube-api-access-glz58\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.619515 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.619388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2a17fb-fce3-4412-b60f-28d92dceb71e-openshift-service-ca\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.619515 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.619441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.720796 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.720714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glz58\" (UniqueName: \"kubernetes.io/projected/cb2a17fb-fce3-4412-b60f-28d92dceb71e-kube-api-access-glz58\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.720796 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.720791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2a17fb-fce3-4412-b60f-28d92dceb71e-openshift-service-ca\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.720989 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.720831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.720989 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:40.720946    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-serving-cert: secret "maas-api-e2e-api-a-20b742-serving-cert" not found
Jul 19 15:29:40.721073 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:40.721003    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls podName:cb2a17fb-fce3-4412-b60f-28d92dceb71e nodeName:}" failed. No retries permitted until 2026-07-19 15:29:41.220988035 +0000 UTC m=+2562.245997088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls") pod "maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" (UID: "cb2a17fb-fce3-4412-b60f-28d92dceb71e") : secret "maas-api-e2e-api-a-20b742-serving-cert" not found
Jul 19 15:29:40.721422 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.721403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2a17fb-fce3-4412-b60f-28d92dceb71e-openshift-service-ca\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:40.729162 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:40.729130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glz58\" (UniqueName: \"kubernetes.io/projected/cb2a17fb-fce3-4412-b60f-28d92dceb71e-kube-api-access-glz58\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:41.225186 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:41.225149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:41.227575 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:41.227558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls\") pod \"maas-api-e2e-api-a-20b742-5d64fb688b-58kmn\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:41.435548 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:41.435512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:41.574723 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:41.574699    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"]
Jul 19 15:29:41.576839 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:29:41.576810    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2a17fb_fce3_4412_b60f_28d92dceb71e.slice/crio-a10f87269fb5171d75334d77932e3142b9b4879c324eb8b344ec96ffbe22316b WatchSource:0}: Error finding container a10f87269fb5171d75334d77932e3142b9b4879c324eb8b344ec96ffbe22316b: Status 404 returned error can't find the container with id a10f87269fb5171d75334d77932e3142b9b4879c324eb8b344ec96ffbe22316b
Jul 19 15:29:41.820847 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:41.820767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:42.239420 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.239395    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:29:42.336459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336374    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-token\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336431    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-podinfo\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336459 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cbgq\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-kube-api-access-7cbgq\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336648    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istiod-ca-cert\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336698    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-envoy\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-certs\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336766    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-credential-socket\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336998 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-socket\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.336998 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.336898    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-data\") pod \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\" (UID: \"8872a13b-d56a-40d2-9b44-9153a1a64ad5\") "
Jul 19 15:29:42.337116 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337087    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:42.337116 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337102    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:42.337232 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337122    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:29:42.337307 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337287    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.337369 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337301    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:42.337369 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337313    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.337369 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337327    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.337901 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.337874    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-data" (OuterVolumeSpecName: "istio-data") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:42.338725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.338696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:29:42.338725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.338700    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-kube-api-access-7cbgq" (OuterVolumeSpecName: "kube-api-access-7cbgq") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "kube-api-access-7cbgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:42.338725 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.338718    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:42.339115 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.339094    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-token" (OuterVolumeSpecName: "istio-token") pod "8872a13b-d56a-40d2-9b44-9153a1a64ad5" (UID: "8872a13b-d56a-40d2-9b44-9153a1a64ad5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:42.437784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.437755    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.437784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.437779    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.437784 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.437791    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.437969 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.437798    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.437969 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.437807    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cbgq\" (UniqueName: \"kubernetes.io/projected/8872a13b-d56a-40d2-9b44-9153a1a64ad5-kube-api-access-7cbgq\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.437969 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.437815    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8872a13b-d56a-40d2-9b44-9153a1a64ad5-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:29:42.486829 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.486797    2577 generic.go:356] "Generic (PLEG): container finished" podID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerID="f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37" exitCode=0
Jul 19 15:29:42.486980 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.486850    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"
Jul 19 15:29:42.486980 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.486875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" event={"ID":"8872a13b-d56a-40d2-9b44-9153a1a64ad5","Type":"ContainerDied","Data":"f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37"}
Jul 19 15:29:42.486980 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.486929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf" event={"ID":"8872a13b-d56a-40d2-9b44-9153a1a64ad5","Type":"ContainerDied","Data":"9f74f7c2e760c5677204f226a8c245bb9630fb352cf27b9dcd595bc5d288ecb8"}
Jul 19 15:29:42.486980 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.486958    2577 scope.go:117] "RemoveContainer" containerID="f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37"
Jul 19 15:29:42.488520 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.488470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" event={"ID":"cb2a17fb-fce3-4412-b60f-28d92dceb71e","Type":"ContainerStarted","Data":"9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2"}
Jul 19 15:29:42.488605 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.488532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" event={"ID":"cb2a17fb-fce3-4412-b60f-28d92dceb71e","Type":"ContainerStarted","Data":"a10f87269fb5171d75334d77932e3142b9b4879c324eb8b344ec96ffbe22316b"}
Jul 19 15:29:42.496045 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.496029    2577 scope.go:117] "RemoveContainer" containerID="d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"
Jul 19 15:29:42.503599 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.503581    2577 scope.go:117] "RemoveContainer" containerID="f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37"
Jul 19 15:29:42.503837 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:42.503818    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37\": container with ID starting with f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37 not found: ID does not exist" containerID="f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37"
Jul 19 15:29:42.503879 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.503846    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37"} err="failed to get container status \"f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37\": rpc error: code = NotFound desc = could not find container \"f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37\": container with ID starting with f5e23040664b7ee447f6061352c9626665bc2d80881c737cf09bde2737420e37 not found: ID does not exist"
Jul 19 15:29:42.503879 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.503863    2577 scope.go:117] "RemoveContainer" containerID="d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"
Jul 19 15:29:42.504094 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:29:42.504076    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c\": container with ID starting with d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c not found: ID does not exist" containerID="d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"
Jul 19 15:29:42.504143 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.504100    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c"} err="failed to get container status \"d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c\": rpc error: code = NotFound desc = could not find container \"d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c\": container with ID starting with d4e3cbe9179699a37e56973ddb6a306494af0d532102e46eb40bbc9d1ba0277c not found: ID does not exist"
Jul 19 15:29:42.506920 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.506885    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" podStartSLOduration=2.106421138 podStartE2EDuration="2.506874438s" totalImagesPullingTime="400.4533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:29:42.505376248 +0000 UTC m=+2563.530385321" watchObservedRunningTime="2026-07-19 15:29:42.506874438 +0000 UTC m=+2563.531883512"
Jul 19 15:29:42.519734 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.519708    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"]
Jul 19 15:29:42.523575 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:42.523554    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-42a367-openshift-default-55755f8989-q58zf"]
Jul 19 15:29:43.495422 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:43.495385    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:43.539431 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:43.539400    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" path="/var/lib/kubelet/pods/8872a13b-d56a-40d2-9b44-9153a1a64ad5/volumes"
Jul 19 15:29:44.969716 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:44.969686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:45.465433 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.465395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:45.749773 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.749740    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"]
Jul 19 15:29:45.750319 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.750295    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy"
Jul 19 15:29:45.750485 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.750469    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy"
Jul 19 15:29:45.750651 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.750636    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy"
Jul 19 15:29:45.750741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.750730    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy"
Jul 19 15:29:45.750936 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.750922    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy"
Jul 19 15:29:45.751055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.751041    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8872a13b-d56a-40d2-9b44-9153a1a64ad5" containerName="istio-proxy"
Jul 19 15:29:45.755688 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.755668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.758098 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.758075    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-accabb-openshift-default-dockercfg-swthn\""
Jul 19 15:29:45.766206 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.766179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"]
Jul 19 15:29:45.773835 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.773810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:45.869676 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbctx\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-kube-api-access-zbctx\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.869866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-socket\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.869866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-certs\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.869866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-podinfo\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.869866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-envoy\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.869866 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-data\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.870055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-token\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.870055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istiod-ca-cert\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.870055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.869952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-credential-socket\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.970985 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.970950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-podinfo\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-envoy\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-data\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-token\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istiod-ca-cert\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-credential-socket\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbctx\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-kube-api-access-zbctx\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-socket\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971405 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-certs\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971864 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-credential-socket\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971864 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-socket\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971864 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-certs\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971864 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-data\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.971864 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.971838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istiod-ca-cert\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.973597 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.973571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-envoy\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.973791 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.973770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-podinfo\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.978907 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.978885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-token\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:45.979419 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:45.979397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbctx\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-kube-api-access-zbctx\") pod \"e2e-api-b-accabb-openshift-default-58df4796b9-khg8t\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") " pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:46.069121 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.069036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:46.238288 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.236390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"]
Jul 19 15:29:46.244965 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:29:46.244939    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7292fb7_4daa_44e0_ae16_eaf843442bf5.slice/crio-97b1e2938eec80db9e5a69ec55c104f25a0cbfc6513bb1bdf28d323fc737b676 WatchSource:0}: Error finding container 97b1e2938eec80db9e5a69ec55c104f25a0cbfc6513bb1bdf28d323fc737b676: Status 404 returned error can't find the container with id 97b1e2938eec80db9e5a69ec55c104f25a0cbfc6513bb1bdf28d323fc737b676
Jul 19 15:29:46.247320 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.247285    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:46.247446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.247377    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:46.247446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.247422    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:46.288483 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.288464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:46.512078 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.512037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" event={"ID":"b7292fb7-4daa-44e0-ae16-eaf843442bf5","Type":"ContainerStarted","Data":"d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"}
Jul 19 15:29:46.512231 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:46.512097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" event={"ID":"b7292fb7-4daa-44e0-ae16-eaf843442bf5","Type":"ContainerStarted","Data":"97b1e2938eec80db9e5a69ec55c104f25a0cbfc6513bb1bdf28d323fc737b676"}
Jul 19 15:29:47.069886 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:47.069841    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:47.075041 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:47.075014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:47.097835 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:47.097778    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podStartSLOduration=2.097762291 podStartE2EDuration="2.097762291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:29:46.533887704 +0000 UTC m=+2567.558896781" watchObservedRunningTime="2026-07-19 15:29:47.097762291 +0000 UTC m=+2568.122771365"
Jul 19 15:29:47.521015 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:47.520980    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:47.526223 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:47.526198    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:29:48.452736 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.452693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:48.530712 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.530678    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"]
Jul 19 15:29:48.534934 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.534874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.537803 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.537780    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-accabb-serving-cert\""
Jul 19 15:29:48.547049 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.547026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"]
Jul 19 15:29:48.556813 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.556767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:48.623652 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.623619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:48.699468 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.699419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkknz\" (UniqueName: \"kubernetes.io/projected/604a1dea-1e4e-41ea-8d03-791edda5948c-kube-api-access-mkknz\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.699681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.699482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604a1dea-1e4e-41ea-8d03-791edda5948c-maas-api-tls\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.699681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.699535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604a1dea-1e4e-41ea-8d03-791edda5948c-openshift-service-ca\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.727569 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.727537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:48.800551 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.800476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkknz\" (UniqueName: \"kubernetes.io/projected/604a1dea-1e4e-41ea-8d03-791edda5948c-kube-api-access-mkknz\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.800719 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.800580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604a1dea-1e4e-41ea-8d03-791edda5948c-maas-api-tls\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.800719 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.800608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604a1dea-1e4e-41ea-8d03-791edda5948c-openshift-service-ca\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.801425 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.801399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604a1dea-1e4e-41ea-8d03-791edda5948c-openshift-service-ca\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.803160 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.803134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604a1dea-1e4e-41ea-8d03-791edda5948c-maas-api-tls\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.808830 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.808802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkknz\" (UniqueName: \"kubernetes.io/projected/604a1dea-1e4e-41ea-8d03-791edda5948c-kube-api-access-mkknz\") pod \"maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.849699 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.849667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:48.994585 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:48.994561    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"]
Jul 19 15:29:48.997147 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:29:48.997110    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod604a1dea_1e4e_41ea_8d03_791edda5948c.slice/crio-4a2b7612fec07d31fa36b11f0e6ab7e71cf9fb363384c3abcf34acd30147e790 WatchSource:0}: Error finding container 4a2b7612fec07d31fa36b11f0e6ab7e71cf9fb363384c3abcf34acd30147e790: Status 404 returned error can't find the container with id 4a2b7612fec07d31fa36b11f0e6ab7e71cf9fb363384c3abcf34acd30147e790
Jul 19 15:29:49.119570 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:49.119529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:49.219663 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:49.219627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:49.506789 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:49.506758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:29:49.546710 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:49.546581    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6" event={"ID":"604a1dea-1e4e-41ea-8d03-791edda5948c","Type":"ContainerStarted","Data":"87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023"}
Jul 19 15:29:49.546710 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:49.546637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6" event={"ID":"604a1dea-1e4e-41ea-8d03-791edda5948c","Type":"ContainerStarted","Data":"4a2b7612fec07d31fa36b11f0e6ab7e71cf9fb363384c3abcf34acd30147e790"}
Jul 19 15:29:49.563851 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:49.563803    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6" podStartSLOduration=1.2789289799999999 podStartE2EDuration="1.563785206s" totalImagesPullingTime="284.856226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:29:49.561414961 +0000 UTC m=+2570.586424039" watchObservedRunningTime="2026-07-19 15:29:49.563785206 +0000 UTC m=+2570.588794280"
Jul 19 15:29:50.551556 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:50.551520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:51.564233 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:51.564158    2577 generic.go:356] "Generic (PLEG): container finished" podID="10121645-966b-42a4-b65a-d800bad6b04a" containerID="783c5865339e44f7896aa9909da44e47576221adad2e07a4deea88e8c23c3fd1" exitCode=0
Jul 19 15:29:51.564741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:51.564236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" event={"ID":"10121645-966b-42a4-b65a-d800bad6b04a","Type":"ContainerDied","Data":"783c5865339e44f7896aa9909da44e47576221adad2e07a4deea88e8c23c3fd1"}
Jul 19 15:29:51.564741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:51.564702    2577 scope.go:117] "RemoveContainer" containerID="783c5865339e44f7896aa9909da44e47576221adad2e07a4deea88e8c23c3fd1"
Jul 19 15:29:51.565659 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:51.565631    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:51.565779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:51.565709    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:51.565779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:51.565754    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:29:52.571354 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:52.571321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" event={"ID":"10121645-966b-42a4-b65a-d800bad6b04a","Type":"ContainerStarted","Data":"9301753b0ed4d63ae78b16acc6a19b03c2f57f99b43e7779fe9a4e7d58fce595"}
Jul 19 15:29:52.968749 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:52.968655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:29:53.209406 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:53.209373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:53.210568 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:53.210539    2577 patch_prober.go:28] interesting pod/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 19 15:29:53.210704 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:53.210588    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 19 15:29:54.209335 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:54.209292    2577 patch_prober.go:28] interesting pod/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 19 15:29:54.209753 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:54.209351    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 19 15:29:54.470648 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:54.470563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:55.209865 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:55.209830    2577 patch_prober.go:28] interesting pod/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 19 15:29:55.210261 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:55.209895    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 19 15:29:56.209805 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:56.209772    2577 patch_prober.go:28] interesting pod/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 19 15:29:56.209977 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:56.209824    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 19 15:29:56.570251 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:56.570223    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:29:57.214366 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:57.214325    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:57.217348 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:57.217287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:29:59.769810 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:29:59.769781    2577 scope.go:117] "RemoveContainer" containerID="3d4271c4af1fa5d7b012ed93f59ddb910bf71f4915b6720e0fd2b2e3f3b2b89e"
Jul 19 15:30:01.684607 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:01.684572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:01.785067 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:01.785029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:03.561646 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:03.561608    2577 patch_prober.go:28] interesting pod/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 19 15:30:03.562106 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:03.561689    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 19 15:30:04.635162 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:04.635130    2577 generic.go:356] "Generic (PLEG): container finished" podID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerID="d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1" exitCode=0
Jul 19 15:30:04.635606 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:04.635192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" event={"ID":"b7292fb7-4daa-44e0-ae16-eaf843442bf5","Type":"ContainerDied","Data":"d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"}
Jul 19 15:30:04.635606 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:04.635578    2577 scope.go:117] "RemoveContainer" containerID="d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"
Jul 19 15:30:04.636615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:04.636589    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:30:04.636678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:04.636657    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:30:04.636714 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:04.636695    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:30:05.641990 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:05.641955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" event={"ID":"b7292fb7-4daa-44e0-ae16-eaf843442bf5","Type":"ContainerStarted","Data":"35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727"}
Jul 19 15:30:06.069653 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:06.069620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:30:06.069924 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:06.069904    2577 patch_prober.go:28] interesting pod/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 19 15:30:06.069978 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:06.069957    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 19 15:30:07.069373 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:07.069336    2577 patch_prober.go:28] interesting pod/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 19 15:30:07.069835 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:07.069397    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 19 15:30:08.070343 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:08.070307    2577 patch_prober.go:28] interesting pod/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 19 15:30:08.070778 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:08.070363    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 19 15:30:09.069581 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:09.069546    2577 patch_prober.go:28] interesting pod/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 19 15:30:09.069766 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:09.069602    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 19 15:30:10.030788 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:10.030748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:10.072656 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:10.072627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:30:10.073101 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:10.073081    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:30:10.073799 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:10.073779    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:30:10.095658 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:10.095628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:11.777391 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:11.777357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:11.877532 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:11.877478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:17.006325 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.006292    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"]
Jul 19 15:30:17.006812 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.006540    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" podUID="cb2a17fb-fce3-4412-b60f-28d92dceb71e" containerName="maas-api" containerID="cri-o://9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2" gracePeriod=30
Jul 19 15:30:17.048170 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.048138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:17.260206 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.260146    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:30:17.292458 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.292431    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls\") pod \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") "
Jul 19 15:30:17.292636 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.292468    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glz58\" (UniqueName: \"kubernetes.io/projected/cb2a17fb-fce3-4412-b60f-28d92dceb71e-kube-api-access-glz58\") pod \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") "
Jul 19 15:30:17.292636 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.292524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2a17fb-fce3-4412-b60f-28d92dceb71e-openshift-service-ca\") pod \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\" (UID: \"cb2a17fb-fce3-4412-b60f-28d92dceb71e\") "
Jul 19 15:30:17.292890 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.292861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2a17fb-fce3-4412-b60f-28d92dceb71e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cb2a17fb-fce3-4412-b60f-28d92dceb71e" (UID: "cb2a17fb-fce3-4412-b60f-28d92dceb71e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:30:17.294609 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.294582    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2a17fb-fce3-4412-b60f-28d92dceb71e-kube-api-access-glz58" (OuterVolumeSpecName: "kube-api-access-glz58") pod "cb2a17fb-fce3-4412-b60f-28d92dceb71e" (UID: "cb2a17fb-fce3-4412-b60f-28d92dceb71e"). InnerVolumeSpecName "kube-api-access-glz58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:30:17.294609 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.294599    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cb2a17fb-fce3-4412-b60f-28d92dceb71e" (UID: "cb2a17fb-fce3-4412-b60f-28d92dceb71e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:30:17.393381 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.393349    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2a17fb-fce3-4412-b60f-28d92dceb71e-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:17.393381 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.393379    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb2a17fb-fce3-4412-b60f-28d92dceb71e-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:17.393570 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.393389    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glz58\" (UniqueName: \"kubernetes.io/projected/cb2a17fb-fce3-4412-b60f-28d92dceb71e-kube-api-access-glz58\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:17.702656 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.702624    2577 generic.go:356] "Generic (PLEG): container finished" podID="cb2a17fb-fce3-4412-b60f-28d92dceb71e" containerID="9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2" exitCode=0
Jul 19 15:30:17.702828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.702668    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"
Jul 19 15:30:17.702828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.702706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" event={"ID":"cb2a17fb-fce3-4412-b60f-28d92dceb71e","Type":"ContainerDied","Data":"9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2"}
Jul 19 15:30:17.702828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.702760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn" event={"ID":"cb2a17fb-fce3-4412-b60f-28d92dceb71e","Type":"ContainerDied","Data":"a10f87269fb5171d75334d77932e3142b9b4879c324eb8b344ec96ffbe22316b"}
Jul 19 15:30:17.702828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.702784    2577 scope.go:117] "RemoveContainer" containerID="9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2"
Jul 19 15:30:17.713613 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.713592    2577 scope.go:117] "RemoveContainer" containerID="9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2"
Jul 19 15:30:17.713902 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:30:17.713883    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2\": container with ID starting with 9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2 not found: ID does not exist" containerID="9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2"
Jul 19 15:30:17.713988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.713914    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2"} err="failed to get container status \"9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2\": rpc error: code = NotFound desc = could not find container \"9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2\": container with ID starting with 9deebd1706cc39170406da70ca08316736d9844f7e1b780e5206f268b82fd4d2 not found: ID does not exist"
Jul 19 15:30:17.720538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.720506    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"]
Jul 19 15:30:17.724415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:17.724391    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-20b742-5d64fb688b-58kmn"]
Jul 19 15:30:19.538891 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:19.538860    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2a17fb-fce3-4412-b60f-28d92dceb71e" path="/var/lib/kubelet/pods/cb2a17fb-fce3-4412-b60f-28d92dceb71e/volumes"
Jul 19 15:30:21.200683 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:21.200651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:21.593770 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:21.593738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:27.324568 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:27.324529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:27.391323 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:27.391287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:27.570864 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:27.570831    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"]
Jul 19 15:30:27.571370 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:27.571090    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy" containerID="cri-o://9301753b0ed4d63ae78b16acc6a19b03c2f57f99b43e7779fe9a4e7d58fce595" gracePeriod=30
Jul 19 15:30:27.593474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:27.593356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:27.871937 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:27.871843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:28.470002 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:28.469968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:28.570702 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:28.570669    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:32.779235 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.779204    2577 generic.go:356] "Generic (PLEG): container finished" podID="10121645-966b-42a4-b65a-d800bad6b04a" containerID="9301753b0ed4d63ae78b16acc6a19b03c2f57f99b43e7779fe9a4e7d58fce595" exitCode=0
Jul 19 15:30:32.779615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.779270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" event={"ID":"10121645-966b-42a4-b65a-d800bad6b04a","Type":"ContainerDied","Data":"9301753b0ed4d63ae78b16acc6a19b03c2f57f99b43e7779fe9a4e7d58fce595"}
Jul 19 15:30:32.779615 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.779330    2577 scope.go:117] "RemoveContainer" containerID="783c5865339e44f7896aa9909da44e47576221adad2e07a4deea88e8c23c3fd1"
Jul 19 15:30:32.824942 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.824923    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:30:32.933746 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933673    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-data\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.933746 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933711    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-certs\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.933746 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-envoy\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-credential-socket\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933789    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-socket\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933818    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10121645-966b-42a4-b65a-d800bad6b04a-istio-podinfo\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933832    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-istio-token\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933864    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xbjc\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-kube-api-access-2xbjc\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933927    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10121645-966b-42a4-b65a-d800bad6b04a-istiod-ca-cert\") pod \"10121645-966b-42a4-b65a-d800bad6b04a\" (UID: \"10121645-966b-42a4-b65a-d800bad6b04a\") "
Jul 19 15:30:32.934024 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.933987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:32.934356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.934256    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:32.934356 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.934256    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:32.934460 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.934363    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10121645-966b-42a4-b65a-d800bad6b04a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:30:32.934555 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.934454    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:32.934874 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.934846    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-data" (OuterVolumeSpecName: "istio-data") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:32.935974 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.935951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:32.936162 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.936144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-kube-api-access-2xbjc" (OuterVolumeSpecName: "kube-api-access-2xbjc") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "kube-api-access-2xbjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:30:32.936424 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.936407    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10121645-966b-42a4-b65a-d800bad6b04a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:30:32.936481 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:32.936422    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-istio-token" (OuterVolumeSpecName: "istio-token") pod "10121645-966b-42a4-b65a-d800bad6b04a" (UID: "10121645-966b-42a4-b65a-d800bad6b04a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:30:33.035581 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035553    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035581 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035577    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035581 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035585    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035595    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10121645-966b-42a4-b65a-d800bad6b04a-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035604    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10121645-966b-42a4-b65a-d800bad6b04a-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035613    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035621    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xbjc\" (UniqueName: \"kubernetes.io/projected/10121645-966b-42a4-b65a-d800bad6b04a-kube-api-access-2xbjc\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.035775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.035630    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10121645-966b-42a4-b65a-d800bad6b04a-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:33.785870 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.785841    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"
Jul 19 15:30:33.786340 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.785864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq" event={"ID":"10121645-966b-42a4-b65a-d800bad6b04a","Type":"ContainerDied","Data":"a06c0a4a24a79b577c6bc8985f72c611acdafcda33ad8f13921b469b2d17f738"}
Jul 19 15:30:33.786340 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.785927    2577 scope.go:117] "RemoveContainer" containerID="9301753b0ed4d63ae78b16acc6a19b03c2f57f99b43e7779fe9a4e7d58fce595"
Jul 19 15:30:33.802415 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.802361    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"]
Jul 19 15:30:33.809265 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:33.809244    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-20b742-openshift-default-7dd5cb6b9c-k4grq"]
Jul 19 15:30:35.539765 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:35.539734    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10121645-966b-42a4-b65a-d800bad6b04a" path="/var/lib/kubelet/pods/10121645-966b-42a4-b65a-d800bad6b04a/volumes"
Jul 19 15:30:43.451626 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.451585    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"]
Jul 19 15:30:43.452141 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.451848    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6" podUID="604a1dea-1e4e-41ea-8d03-791edda5948c" containerName="maas-api" containerID="cri-o://87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023" gracePeriod=30
Jul 19 15:30:43.480740 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.480710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:43.707328 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.707270    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:30:43.830036 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.830006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604a1dea-1e4e-41ea-8d03-791edda5948c-maas-api-tls\") pod \"604a1dea-1e4e-41ea-8d03-791edda5948c\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") "
Jul 19 15:30:43.830184 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.830120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkknz\" (UniqueName: \"kubernetes.io/projected/604a1dea-1e4e-41ea-8d03-791edda5948c-kube-api-access-mkknz\") pod \"604a1dea-1e4e-41ea-8d03-791edda5948c\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") "
Jul 19 15:30:43.830184 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.830138    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604a1dea-1e4e-41ea-8d03-791edda5948c-openshift-service-ca\") pod \"604a1dea-1e4e-41ea-8d03-791edda5948c\" (UID: \"604a1dea-1e4e-41ea-8d03-791edda5948c\") "
Jul 19 15:30:43.830536 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.830483    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/604a1dea-1e4e-41ea-8d03-791edda5948c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "604a1dea-1e4e-41ea-8d03-791edda5948c" (UID: "604a1dea-1e4e-41ea-8d03-791edda5948c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:30:43.832160 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.832128    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604a1dea-1e4e-41ea-8d03-791edda5948c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "604a1dea-1e4e-41ea-8d03-791edda5948c" (UID: "604a1dea-1e4e-41ea-8d03-791edda5948c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:30:43.832286 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.832222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604a1dea-1e4e-41ea-8d03-791edda5948c-kube-api-access-mkknz" (OuterVolumeSpecName: "kube-api-access-mkknz") pod "604a1dea-1e4e-41ea-8d03-791edda5948c" (UID: "604a1dea-1e4e-41ea-8d03-791edda5948c"). InnerVolumeSpecName "kube-api-access-mkknz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:30:43.836628 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.836607    2577 generic.go:356] "Generic (PLEG): container finished" podID="604a1dea-1e4e-41ea-8d03-791edda5948c" containerID="87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023" exitCode=0
Jul 19 15:30:43.836722 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.836653    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"
Jul 19 15:30:43.836722 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.836695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6" event={"ID":"604a1dea-1e4e-41ea-8d03-791edda5948c","Type":"ContainerDied","Data":"87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023"}
Jul 19 15:30:43.836845 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.836756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6" event={"ID":"604a1dea-1e4e-41ea-8d03-791edda5948c","Type":"ContainerDied","Data":"4a2b7612fec07d31fa36b11f0e6ab7e71cf9fb363384c3abcf34acd30147e790"}
Jul 19 15:30:43.836845 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.836791    2577 scope.go:117] "RemoveContainer" containerID="87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023"
Jul 19 15:30:43.852572 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.852555    2577 scope.go:117] "RemoveContainer" containerID="87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023"
Jul 19 15:30:43.852835 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:30:43.852815    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023\": container with ID starting with 87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023 not found: ID does not exist" containerID="87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023"
Jul 19 15:30:43.852930 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.852841    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023"} err="failed to get container status \"87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023\": rpc error: code = NotFound desc = could not find container \"87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023\": container with ID starting with 87607ab0c344c4e1d98ed61f67a8a4aa9f5f7ca1c66128935fbc104ddd700023 not found: ID does not exist"
Jul 19 15:30:43.860593 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.860574    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"]
Jul 19 15:30:43.864447 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.864422    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-accabb-5dc6bd4b6b-9wjv6"]
Jul 19 15:30:43.931026 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.931001    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/604a1dea-1e4e-41ea-8d03-791edda5948c-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:43.931026 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.931021    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkknz\" (UniqueName: \"kubernetes.io/projected/604a1dea-1e4e-41ea-8d03-791edda5948c-kube-api-access-mkknz\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:43.931166 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:43.931031    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/604a1dea-1e4e-41ea-8d03-791edda5948c-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:45.538998 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:45.538970    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="604a1dea-1e4e-41ea-8d03-791edda5948c" path="/var/lib/kubelet/pods/604a1dea-1e4e-41ea-8d03-791edda5948c/volumes"
Jul 19 15:30:48.067150 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:48.067112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:48.469600 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:48.469487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:54.189656 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:54.189621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:54.288834 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:54.288795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:54.446000 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:54.445924    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"]
Jul 19 15:30:54.446196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:54.446172    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" containerID="cri-o://35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727" gracePeriod=30
Jul 19 15:30:54.468872 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:54.468838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:54.700782 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:54.700691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:55.069457 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.069419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:55.075170 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.075142    2577 patch_prober.go:28] interesting pod/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:30:55.075305 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.075194    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:30:55.370931 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.370854    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"]
Jul 19 15:30:55.371392 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371377    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy"
Jul 19 15:30:55.371443 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371396    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy"
Jul 19 15:30:55.371443 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371436    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb2a17fb-fce3-4412-b60f-28d92dceb71e" containerName="maas-api"
Jul 19 15:30:55.371528 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371445    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2a17fb-fce3-4412-b60f-28d92dceb71e" containerName="maas-api"
Jul 19 15:30:55.371528 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371460    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="604a1dea-1e4e-41ea-8d03-791edda5948c" containerName="maas-api"
Jul 19 15:30:55.371528 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371469    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="604a1dea-1e4e-41ea-8d03-791edda5948c" containerName="maas-api"
Jul 19 15:30:55.371633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371556    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="604a1dea-1e4e-41ea-8d03-791edda5948c" containerName="maas-api"
Jul 19 15:30:55.371633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371569    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy"
Jul 19 15:30:55.371633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy"
Jul 19 15:30:55.371633 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.371592    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb2a17fb-fce3-4412-b60f-28d92dceb71e" containerName="maas-api"
Jul 19 15:30:55.376211 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.376190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.378544 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.378525    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4bc178-openshift-default-dockercfg-ths7g\""
Jul 19 15:30:55.386074 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.386041    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"]
Jul 19 15:30:55.390699 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.390671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:55.423877 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.423851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkvt5\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-kube-api-access-qkvt5\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424000 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.423890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-envoy\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424000 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.423916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-podinfo\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424000 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.423972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-credential-socket\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424000 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.423996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-data\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.424021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-socket\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.424142    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-token\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.424187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-certs\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.424334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.424229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istiod-ca-cert\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.524848 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.524810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkvt5\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-kube-api-access-qkvt5\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525045 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.524854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-envoy\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525045 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.524979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-podinfo\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525174 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-credential-socket\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525174 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-data\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525174 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-socket\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525329 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-token\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525329 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-certs\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525329 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istiod-ca-cert\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525508 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-data\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525568 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-socket\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525635 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-certs\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525752 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-credential-socket\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.525939 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.525921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istiod-ca-cert\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.527842 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.527820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-envoy\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.527985 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.527966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-podinfo\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.533602 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.533581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-token\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.533724 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.533648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkvt5\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-kube-api-access-qkvt5\") pod \"e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") " pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.673690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.673601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:55.688392 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.688362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:55.825640 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.825612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"]
Jul 19 15:30:55.826869 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:30:55.826843    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55a53e0_83f1_42fa_bf4c_7e3f19937890.slice/crio-582944144d3e5fe695d861f784001c153b5acb62139c4b5017c8b022f97d0983 WatchSource:0}: Error finding container 582944144d3e5fe695d861f784001c153b5acb62139c4b5017c8b022f97d0983: Status 404 returned error can't find the container with id 582944144d3e5fe695d861f784001c153b5acb62139c4b5017c8b022f97d0983
Jul 19 15:30:55.828970 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.828936    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:30:55.829043 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.829018    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:30:55.829081 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.829066    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:30:55.897099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.897071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" event={"ID":"c55a53e0-83f1-42fa-bf4c-7e3f19937890","Type":"ContainerStarted","Data":"c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"}
Jul 19 15:30:55.897201 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.897121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" event={"ID":"c55a53e0-83f1-42fa-bf4c-7e3f19937890","Type":"ContainerStarted","Data":"582944144d3e5fe695d861f784001c153b5acb62139c4b5017c8b022f97d0983"}
Jul 19 15:30:55.916593 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:55.916540    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podStartSLOduration=0.916520664 podStartE2EDuration="916.520664ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:30:55.913774548 +0000 UTC m=+2636.938783626" watchObservedRunningTime="2026-07-19 15:30:55.916520664 +0000 UTC m=+2636.941529740"
Jul 19 15:30:56.689190 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:56.689151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:56.694211 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:56.694184    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:56.904411 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:56.904379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:56.905357 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:56.905330    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:30:57.912789 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:57.912752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:58.042279 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.042247    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"]
Jul 19 15:30:58.042737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.042715    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy"
Jul 19 15:30:58.042737 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.042736    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10121645-966b-42a4-b65a-d800bad6b04a" containerName="istio-proxy"
Jul 19 15:30:58.045937 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.045911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.048446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.048422    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-4bc178-serving-cert\""
Jul 19 15:30:58.053678 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.053658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:58.058568 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.058546    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"]
Jul 19 15:30:58.143691 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.143652    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:58.149732 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.149698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rv6w\" (UniqueName: \"kubernetes.io/projected/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-kube-api-access-4rv6w\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.149732 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.149731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.149916 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.149831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-maas-api-tls\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.246123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.246090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:58.250807 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.250784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-maas-api-tls\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.250903 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.250834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rv6w\" (UniqueName: \"kubernetes.io/projected/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-kube-api-access-4rv6w\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.250903 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.250860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.251598 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.251571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.253390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.253369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-maas-api-tls\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.263129 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.263103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rv6w\" (UniqueName: \"kubernetes.io/projected/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-kube-api-access-4rv6w\") pod \"maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.359544 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.359481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.496544 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.496451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"]
Jul 19 15:30:58.498169 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:30:58.498141    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2888aff3_e6d1_4f87_a85e_5a9384e3bdd8.slice/crio-dbfb65657105072954846ffd8994ba0101e1ea9b8ea743846227367c67f38f10 WatchSource:0}: Error finding container dbfb65657105072954846ffd8994ba0101e1ea9b8ea743846227367c67f38f10: Status 404 returned error can't find the container with id dbfb65657105072954846ffd8994ba0101e1ea9b8ea743846227367c67f38f10
Jul 19 15:30:58.657458 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.657421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:58.760165 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.760068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:30:58.917874 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.917798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" event={"ID":"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8","Type":"ContainerStarted","Data":"f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc"}
Jul 19 15:30:58.918329 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.918228    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:30:58.918329 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.918257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" event={"ID":"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8","Type":"ContainerStarted","Data":"dbfb65657105072954846ffd8994ba0101e1ea9b8ea743846227367c67f38f10"}
Jul 19 15:30:58.935665 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:58.935613    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" podStartSLOduration=1.5958383189999998 podStartE2EDuration="1.935596036s" totalImagesPullingTime="339.757717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:30:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:30:58.93380551 +0000 UTC m=+2639.958814619" watchObservedRunningTime="2026-07-19 15:30:58.935596036 +0000 UTC m=+2639.960605110"
Jul 19 15:30:59.705474 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.705448    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:30:59.765449 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765419    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbctx\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-kube-api-access-zbctx\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765449 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765452    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-token\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765480    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-socket\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765555    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-podinfo\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765570    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-data\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765586    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istiod-ca-cert\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765606    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-envoy\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765631    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-certs\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.765741 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-credential-socket\") pod \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\" (UID: \"b7292fb7-4daa-44e0-ae16-eaf843442bf5\") "
Jul 19 15:30:59.766100 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.765923    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:59.766100 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.766056    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.766209 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.766126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:59.766284 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.766266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:59.766377 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.766340    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:30:59.766559 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.766541    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:59.767955 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.767927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-kube-api-access-zbctx" (OuterVolumeSpecName: "kube-api-access-zbctx") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "kube-api-access-zbctx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:30:59.767955 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.767944    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:30:59.768182 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.768164    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:30:59.768235 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.768191    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7292fb7-4daa-44e0-ae16-eaf843442bf5" (UID: "b7292fb7-4daa-44e0-ae16-eaf843442bf5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:30:59.866538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866481    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866534    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866538 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866544    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866553    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbctx\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-kube-api-access-zbctx\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866562    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866570    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866577    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.866775 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.866587    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7292fb7-4daa-44e0-ae16-eaf843442bf5-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:30:59.924650 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.924615    2577 generic.go:356] "Generic (PLEG): container finished" podID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerID="35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727" exitCode=0
Jul 19 15:30:59.925064 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.924665    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"
Jul 19 15:30:59.925064 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.924704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" event={"ID":"b7292fb7-4daa-44e0-ae16-eaf843442bf5","Type":"ContainerDied","Data":"35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727"}
Jul 19 15:30:59.925064 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.924754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t" event={"ID":"b7292fb7-4daa-44e0-ae16-eaf843442bf5","Type":"ContainerDied","Data":"97b1e2938eec80db9e5a69ec55c104f25a0cbfc6513bb1bdf28d323fc737b676"}
Jul 19 15:30:59.925064 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.924780    2577 scope.go:117] "RemoveContainer" containerID="35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727"
Jul 19 15:30:59.933968 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.933949    2577 scope.go:117] "RemoveContainer" containerID="d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"
Jul 19 15:30:59.941986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.941968    2577 scope.go:117] "RemoveContainer" containerID="35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727"
Jul 19 15:30:59.942240 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:30:59.942215    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727\": container with ID starting with 35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727 not found: ID does not exist" containerID="35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727"
Jul 19 15:30:59.942318 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.942253    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727"} err="failed to get container status \"35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727\": rpc error: code = NotFound desc = could not find container \"35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727\": container with ID starting with 35b2fc670c447eba30918b8ded57a8c75fd31d3763191d85b8332a1a7e290727 not found: ID does not exist"
Jul 19 15:30:59.942318 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.942278    2577 scope.go:117] "RemoveContainer" containerID="d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"
Jul 19 15:30:59.942530 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:30:59.942488    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1\": container with ID starting with d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1 not found: ID does not exist" containerID="d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"
Jul 19 15:30:59.942577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.942539    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1"} err="failed to get container status \"d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1\": rpc error: code = NotFound desc = could not find container \"d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1\": container with ID starting with d923dd5dfad1c7be806577815f23da04d70da364c0bf3496332b8872fd5343d1 not found: ID does not exist"
Jul 19 15:30:59.946369 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.946345    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"]
Jul 19 15:30:59.950612 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:30:59.950592    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-accabb-openshift-default-58df4796b9-khg8t"]
Jul 19 15:31:01.539302 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:01.539258    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" path="/var/lib/kubelet/pods/b7292fb7-4daa-44e0-ae16-eaf843442bf5/volumes"
Jul 19 15:31:02.386102 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:02.386066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:02.747246 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:02.747218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:03.046719 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.046641    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"]
Jul 19 15:31:03.047049 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.047033    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy"
Jul 19 15:31:03.047099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.047051    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy"
Jul 19 15:31:03.047099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.047078    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy"
Jul 19 15:31:03.047099 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.047085    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy"
Jul 19 15:31:03.047196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.047140    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy"
Jul 19 15:31:03.047196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.047150    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7292fb7-4daa-44e0-ae16-eaf843442bf5" containerName="istio-proxy"
Jul 19 15:31:03.052028 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.052007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.054369 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.054345    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ccd7ab-openshift-default-dockercfg-cqs8r\""
Jul 19 15:31:03.060245 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.059856    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"]
Jul 19 15:31:03.075779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.075758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:03.098828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.098809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istiod-ca-cert\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.098930 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.098844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-socket\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.098930 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.098862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs5pt\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-kube-api-access-fs5pt\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.099015 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.098935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-certs\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.099015 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.098990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-podinfo\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.099080 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.099014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-credential-socket\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.099080 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.099059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-envoy\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.099160 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.099139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-token\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.099196 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.099182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-data\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.199875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.199835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-data\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200075 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.199884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istiod-ca-cert\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200075 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.199923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-socket\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200075 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.199946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs5pt\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-kube-api-access-fs5pt\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200075 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.199979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-certs\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200075 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-podinfo\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-credential-socket\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-data\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-envoy\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200390 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-token\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200790 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-socket\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200894 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-certs\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.200958 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.200898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-credential-socket\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.201131 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.201107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istiod-ca-cert\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.202687 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.202668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-podinfo\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.202796 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.202768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-envoy\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.207253 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.207230    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-token\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.207664 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.207644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs5pt\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-kube-api-access-fs5pt\") pod \"e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") " pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.364872 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.364797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:03.403245 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.403218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:03.501730 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.501701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"]
Jul 19 15:31:03.503412 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:31:03.503387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5df19f1_366e_4ddc_af95_7cda19d14ccc.slice/crio-d37e750bd5fa21be8d7c511430def07e7abd4891b78ec23befc0f99230468e75 WatchSource:0}: Error finding container d37e750bd5fa21be8d7c511430def07e7abd4891b78ec23befc0f99230468e75: Status 404 returned error can't find the container with id d37e750bd5fa21be8d7c511430def07e7abd4891b78ec23befc0f99230468e75
Jul 19 15:31:03.505984 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.505953    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:03.506072 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.506020    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:03.506072 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.506050    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:03.945107 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.945071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" event={"ID":"a5df19f1-366e-4ddc-af95-7cda19d14ccc","Type":"ContainerStarted","Data":"1b3890f5aa88328202d913412d3845a55284e73432e7f2058c359591c5b833cf"}
Jul 19 15:31:03.945513 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.945123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" event={"ID":"a5df19f1-366e-4ddc-af95-7cda19d14ccc","Type":"ContainerStarted","Data":"d37e750bd5fa21be8d7c511430def07e7abd4891b78ec23befc0f99230468e75"}
Jul 19 15:31:03.963719 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:03.963666    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podStartSLOduration=0.963647961 podStartE2EDuration="963.647961ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:31:03.961913214 +0000 UTC m=+2644.986922289" watchObservedRunningTime="2026-07-19 15:31:03.963647961 +0000 UTC m=+2644.988657037"
Jul 19 15:31:04.365724 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:04.365693    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:04.370278 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:04.370254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:04.950051 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:04.950026    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:04.950988 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:04.950970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:05.715957 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:05.715928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:05.851142 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:05.851098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:05.936186 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:05.936157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:31:05.951531 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:05.951464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:06.051310 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.051277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:06.118637 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.118607    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"]
Jul 19 15:31:06.122830 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.122806    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.125555 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.125533    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ccd7ab-serving-cert\""
Jul 19 15:31:06.132160 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.131800    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"]
Jul 19 15:31:06.166078 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.166050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:06.227523 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.227469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.227708 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.227610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxlzr\" (UniqueName: \"kubernetes.io/projected/22a845fa-b9bb-4285-a8bd-f2966855e038-kube-api-access-qxlzr\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.227708 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.227651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a845fa-b9bb-4285-a8bd-f2966855e038-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.328938 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.328844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxlzr\" (UniqueName: \"kubernetes.io/projected/22a845fa-b9bb-4285-a8bd-f2966855e038-kube-api-access-qxlzr\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.328938 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.328901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a845fa-b9bb-4285-a8bd-f2966855e038-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.329247 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.329222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.329554 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:31:06.329486    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-serving-cert: secret "maas-api-e2e-shared-b-ccd7ab-serving-cert" not found
Jul 19 15:31:06.329696 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:31:06.329614    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls podName:22a845fa-b9bb-4285-a8bd-f2966855e038 nodeName:}" failed. No retries permitted until 2026-07-19 15:31:06.829588327 +0000 UTC m=+2647.854597383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls") pod "maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" (UID: "22a845fa-b9bb-4285-a8bd-f2966855e038") : secret "maas-api-e2e-shared-b-ccd7ab-serving-cert" not found
Jul 19 15:31:06.329845 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.329823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a845fa-b9bb-4285-a8bd-f2966855e038-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.338067 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.338037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxlzr\" (UniqueName: \"kubernetes.io/projected/22a845fa-b9bb-4285-a8bd-f2966855e038-kube-api-access-qxlzr\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.589444 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.589360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:06.690811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.690777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:06.792558 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.792526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:06.834167 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.834116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:06.849249 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:06.849158    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls\") pod \"maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:07.036205 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:07.036148    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:07.196859 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:07.196828    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"]
Jul 19 15:31:07.199046 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:31:07.199022    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a845fa_b9bb_4285_a8bd_f2966855e038.slice/crio-fb2bde46e9c4aec84659da5b706411e3dc4306c1afceb0b7b81d46a9fd146e2d WatchSource:0}: Error finding container fb2bde46e9c4aec84659da5b706411e3dc4306c1afceb0b7b81d46a9fd146e2d: Status 404 returned error can't find the container with id fb2bde46e9c4aec84659da5b706411e3dc4306c1afceb0b7b81d46a9fd146e2d
Jul 19 15:31:07.967595 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:07.967511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" event={"ID":"22a845fa-b9bb-4285-a8bd-f2966855e038","Type":"ContainerStarted","Data":"2efe20c572737725cc819503e8c19219004e10d68dff75ea2e6d7572b9047f2a"}
Jul 19 15:31:07.967595 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:07.967567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" event={"ID":"22a845fa-b9bb-4285-a8bd-f2966855e038","Type":"ContainerStarted","Data":"fb2bde46e9c4aec84659da5b706411e3dc4306c1afceb0b7b81d46a9fd146e2d"}
Jul 19 15:31:07.967595 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:07.967598    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:07.983973 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:07.983925    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" podStartSLOduration=1.667372173 podStartE2EDuration="1.98391062s" totalImagesPullingTime="316.538447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:31:07.982176271 +0000 UTC m=+2649.007185557" watchObservedRunningTime="2026-07-19 15:31:07.98391062 +0000 UTC m=+2649.008919755"
Jul 19 15:31:10.422211 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:10.422179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:12.945637 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:12.945600    2577 patch_prober.go:28] interesting pod/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 19 15:31:12.946015 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:12.945651    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 19 15:31:13.977908 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:13.977878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:31:13.999534 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:13.999470    2577 generic.go:356] "Generic (PLEG): container finished" podID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerID="c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117" exitCode=0
Jul 19 15:31:13.999685 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:13.999544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" event={"ID":"c55a53e0-83f1-42fa-bf4c-7e3f19937890","Type":"ContainerDied","Data":"c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"}
Jul 19 15:31:13.999984 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:13.999966    2577 scope.go:117] "RemoveContainer" containerID="c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"
Jul 19 15:31:14.001121 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:14.001088    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:14.001225 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:14.001171    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:14.001225 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:14.001217    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:15.006940 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:15.006905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" event={"ID":"c55a53e0-83f1-42fa-bf4c-7e3f19937890","Type":"ContainerStarted","Data":"780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2"}
Jul 19 15:31:15.688586 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:15.688553    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:31:15.688768 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:15.688730    2577 patch_prober.go:28] interesting pod/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 19 15:31:15.688811 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:15.688776    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 19 15:31:16.689758 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:16.689723    2577 patch_prober.go:28] interesting pod/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 19 15:31:16.690153 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:16.689780    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 19 15:31:17.689068 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:17.688978    2577 patch_prober.go:28] interesting pod/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 19 15:31:17.689210 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:17.689061    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 19 15:31:18.689558 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:18.689518    2577 patch_prober.go:28] interesting pod/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Jul 19 15:31:18.689986 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:18.689581    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Jul 19 15:31:19.413854 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:19.413814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:19.520431 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:19.520394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:19.692089 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:19.692015    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:31:19.692517 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:19.692370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:31:19.693092 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:19.693072    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:31:20.536056 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:20.536013    2577 patch_prober.go:28] interesting pod/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": read tcp 10.134.0.2:57118->10.134.0.79:15021: read: connection reset by peer" start-of-body=
Jul 19 15:31:20.536222 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:20.536084    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": read tcp 10.134.0.2:57118->10.134.0.79:15021: read: connection reset by peer"
Jul 19 15:31:21.045743 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:21.045708    2577 generic.go:356] "Generic (PLEG): container finished" podID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerID="1b3890f5aa88328202d913412d3845a55284e73432e7f2058c359591c5b833cf" exitCode=0
Jul 19 15:31:21.046231 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:21.045780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" event={"ID":"a5df19f1-366e-4ddc-af95-7cda19d14ccc","Type":"ContainerDied","Data":"1b3890f5aa88328202d913412d3845a55284e73432e7f2058c359591c5b833cf"}
Jul 19 15:31:21.046231 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:21.046144    2577 scope.go:117] "RemoveContainer" containerID="1b3890f5aa88328202d913412d3845a55284e73432e7f2058c359591c5b833cf"
Jul 19 15:31:21.047113 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:21.047087    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:21.047175 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:21.047150    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:21.047215 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:21.047184    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 19 15:31:22.053235 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:22.053195    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" event={"ID":"a5df19f1-366e-4ddc-af95-7cda19d14ccc","Type":"ContainerStarted","Data":"33d2e55034ed3e25ba778396959efc69acbde85672f90ea05ec5c0ccb94d4a8b"}
Jul 19 15:31:22.365726 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:22.365630    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:22.365889 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:22.365863    2577 patch_prober.go:28] interesting pod/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 19 15:31:22.365943 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:22.365925    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 19 15:31:23.365564 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:23.365526    2577 patch_prober.go:28] interesting pod/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 19 15:31:23.365949 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:23.365591    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 19 15:31:24.365354 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:24.365313    2577 patch_prober.go:28] interesting pod/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 19 15:31:24.365585 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:24.365385    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 19 15:31:25.365939 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:25.365898    2577 patch_prober.go:28] interesting pod/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 19 15:31:25.366345 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:25.365979    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 19 15:31:26.368446 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:26.368414    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:26.368881 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:26.368824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:26.369585 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:26.369565    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:31:27.377019 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:27.376987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:27.478970 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:27.478925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:28.480001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:28.479967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:51.847717 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:51.847635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:52.789175 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:52.789129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:31:59.824963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:31:59.824922    2577 scope.go:117] "RemoveContainer" containerID="63656f4b3363dd07e5cba86371d4b506829dfb3618590f23cd386020a1ba0b28"
Jul 19 15:32:42.031940 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:32:42.031902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:33:28.606471 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:33:28.606393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:33:28.707840 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:33:28.707805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:33:52.810427 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:33:52.810382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:33:52.912882 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:33:52.912705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:33:56.190169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:33:56.190139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:33:57.004133 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:33:57.004100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:34:07.394868 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:34:07.394828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:34:36.508137 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:34:36.508100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:34:53.179936 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:34:53.179856    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:35:19.472087 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:35:19.472045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:35:20.266011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:35:20.265982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:35:53.701791 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:35:53.701754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:06.513850 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:06.513810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:06.615260 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:06.615227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:10.695391 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:10.695352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:20.122470 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:20.122433    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:20.221827 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:20.221791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:28.195454 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:28.195374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:28.295673 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:28.295601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:50.072686 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:50.072650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:36:50.891549 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:36:50.891488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:10.771955 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:10.771915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:10.872055 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:10.872024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:20.513283 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:20.513248    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"]
Jul 19 15:37:20.513876 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:20.513533    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" podUID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" containerName="maas-api" containerID="cri-o://f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc" gracePeriod=30
Jul 19 15:37:20.553476 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:20.553442    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:20.970556 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:20.970531    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:37:21.047343 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.047258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rv6w\" (UniqueName: \"kubernetes.io/projected/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-kube-api-access-4rv6w\") pod \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") "
Jul 19 15:37:21.047343 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.047306    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-openshift-service-ca\") pod \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") "
Jul 19 15:37:21.047611 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.047348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-maas-api-tls\") pod \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\" (UID: \"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8\") "
Jul 19 15:37:21.047687 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.047663    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" (UID: "2888aff3-e6d1-4f87-a85e-5a9384e3bdd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:37:21.049418 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.049396    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" (UID: "2888aff3-e6d1-4f87-a85e-5a9384e3bdd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:37:21.049530 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.049488    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-kube-api-access-4rv6w" (OuterVolumeSpecName: "kube-api-access-4rv6w") pod "2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" (UID: "2888aff3-e6d1-4f87-a85e-5a9384e3bdd8"). InnerVolumeSpecName "kube-api-access-4rv6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:21.148168 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.148132    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rv6w\" (UniqueName: \"kubernetes.io/projected/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-kube-api-access-4rv6w\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:21.148168 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.148163    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:21.148168 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.148175    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:21.834322 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.834238    2577 generic.go:356] "Generic (PLEG): container finished" podID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" containerID="f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc" exitCode=0
Jul 19 15:37:21.834322 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.834284    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"
Jul 19 15:37:21.834779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.834322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" event={"ID":"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8","Type":"ContainerDied","Data":"f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc"}
Jul 19 15:37:21.834779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.834381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" event={"ID":"2888aff3-e6d1-4f87-a85e-5a9384e3bdd8","Type":"ContainerDied","Data":"dbfb65657105072954846ffd8994ba0101e1ea9b8ea743846227367c67f38f10"}
Jul 19 15:37:21.834779 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.834410    2577 scope.go:117] "RemoveContainer" containerID="f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc"
Jul 19 15:37:21.843144 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.843127    2577 scope.go:117] "RemoveContainer" containerID="f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc"
Jul 19 15:37:21.843421 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:37:21.843402    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc\": container with ID starting with f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc not found: ID does not exist" containerID="f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc"
Jul 19 15:37:21.843517 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.843429    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc"} err="failed to get container status \"f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc\": rpc error: code = NotFound desc = could not find container \"f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc\": container with ID starting with f5783d2dad3b8de8347747557e64840ae7f3fe98a387b96ffbda460338eef6bc not found: ID does not exist"
Jul 19 15:37:21.854709 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.854687    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"]
Jul 19 15:37:21.858706 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:21.858687    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c"]
Jul 19 15:37:23.540157 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:23.540124    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" path="/var/lib/kubelet/pods/2888aff3-e6d1-4f87-a85e-5a9384e3bdd8/volumes"
Jul 19 15:37:23.931481 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:23.931375    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4bc178-5855bf49c5-rpd6c" podUID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.78:8443/health\": context deadline exceeded"
Jul 19 15:37:25.160989 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:25.160953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:25.569371 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:25.569339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:31.573744 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:31.573714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:31.645161 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:31.645127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:31.858011 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:31.857928    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"]
Jul 19 15:37:31.858159 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:31.858141    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" containerID="cri-o://780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2" gracePeriod=30
Jul 19 15:37:31.875703 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:31.875661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:32.121168 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:32.121080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:32.872090 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:32.872049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:32.972764 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:32.972731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:34.694557 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:34.694512    2577 patch_prober.go:28] interesting pod/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:37:34.694962 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:34.694578    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:37:37.105875 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.105849    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:37:37.193314 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193239    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkvt5\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-kube-api-access-qkvt5\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193314 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193271    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-podinfo\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193314 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193303    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-envoy\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193325    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istiod-ca-cert\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193354    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-token\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193383    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-certs\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-credential-socket\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193426    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-socket\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193604 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193452    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-data\") pod \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\" (UID: \"c55a53e0-83f1-42fa-bf4c-7e3f19937890\") "
Jul 19 15:37:37.193934 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193875    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:37.193934 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.193891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:37:37.194107 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.194075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:37.194192 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.194137    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:37.194570 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.194545    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-data" (OuterVolumeSpecName: "istio-data") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:37.195645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.195612    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:37:37.195645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.195625    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-kube-api-access-qkvt5" (OuterVolumeSpecName: "kube-api-access-qkvt5") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "kube-api-access-qkvt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:37.195645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.195632    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-token" (OuterVolumeSpecName: "istio-token") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:37.195645 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.195641    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c55a53e0-83f1-42fa-bf4c-7e3f19937890" (UID: "c55a53e0-83f1-42fa-bf4c-7e3f19937890"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:37.295091 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295064    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295091 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295088    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295091 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295098    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295106    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295114    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295123    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295132    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295141    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkvt5\" (UniqueName: \"kubernetes.io/projected/c55a53e0-83f1-42fa-bf4c-7e3f19937890-kube-api-access-qkvt5\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.295334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.295149    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c55a53e0-83f1-42fa-bf4c-7e3f19937890-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:37.916612 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.916531    2577 generic.go:356] "Generic (PLEG): container finished" podID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerID="780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2" exitCode=0
Jul 19 15:37:37.916612 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.916574    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"
Jul 19 15:37:37.916802 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.916613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" event={"ID":"c55a53e0-83f1-42fa-bf4c-7e3f19937890","Type":"ContainerDied","Data":"780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2"}
Jul 19 15:37:37.916802 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.916665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp" event={"ID":"c55a53e0-83f1-42fa-bf4c-7e3f19937890","Type":"ContainerDied","Data":"582944144d3e5fe695d861f784001c153b5acb62139c4b5017c8b022f97d0983"}
Jul 19 15:37:37.916802 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.916692    2577 scope.go:117] "RemoveContainer" containerID="780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2"
Jul 19 15:37:37.927417 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.927400    2577 scope.go:117] "RemoveContainer" containerID="c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"
Jul 19 15:37:37.935828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.935803    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"]
Jul 19 15:37:37.936575 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.936556    2577 scope.go:117] "RemoveContainer" containerID="780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2"
Jul 19 15:37:37.936891 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:37:37.936871    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2\": container with ID starting with 780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2 not found: ID does not exist" containerID="780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2"
Jul 19 15:37:37.936958 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.936898    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2"} err="failed to get container status \"780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2\": rpc error: code = NotFound desc = could not find container \"780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2\": container with ID starting with 780acedebe127c4a6947bb3cccf80924d24e6c2504a92ea8b4afbd790ea193a2 not found: ID does not exist"
Jul 19 15:37:37.936958 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.936915    2577 scope.go:117] "RemoveContainer" containerID="c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"
Jul 19 15:37:37.937195 ip-10-0-142-137 kubenswrapper[2577]: E0719 15:37:37.937175    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117\": container with ID starting with c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117 not found: ID does not exist" containerID="c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"
Jul 19 15:37:37.937244 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.937200    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117"} err="failed to get container status \"c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117\": rpc error: code = NotFound desc = could not find container \"c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117\": container with ID starting with c476ce55f007bf929721a96a1868fdaf0895e29c87aab0691ae34763e77a7117 not found: ID does not exist"
Jul 19 15:37:37.938222 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:37.938205    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4bc178-openshift-default-bf8c54cf5-886dp"]
Jul 19 15:37:39.540349 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:39.540318    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" path="/var/lib/kubelet/pods/c55a53e0-83f1-42fa-bf4c-7e3f19937890/volumes"
Jul 19 15:37:42.789681 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:42.789600    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"]
Jul 19 15:37:42.790462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:42.790405    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" podUID="22a845fa-b9bb-4285-a8bd-f2966855e038" containerName="maas-api" containerID="cri-o://2efe20c572737725cc819503e8c19219004e10d68dff75ea2e6d7572b9047f2a" gracePeriod=30
Jul 19 15:37:42.813636 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:42.813592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:42.947335 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:42.947307    2577 generic.go:356] "Generic (PLEG): container finished" podID="22a845fa-b9bb-4285-a8bd-f2966855e038" containerID="2efe20c572737725cc819503e8c19219004e10d68dff75ea2e6d7572b9047f2a" exitCode=0
Jul 19 15:37:42.947487 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:42.947382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" event={"ID":"22a845fa-b9bb-4285-a8bd-f2966855e038","Type":"ContainerDied","Data":"2efe20c572737725cc819503e8c19219004e10d68dff75ea2e6d7572b9047f2a"}
Jul 19 15:37:43.045992 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.045936    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:37:43.141019 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.140990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxlzr\" (UniqueName: \"kubernetes.io/projected/22a845fa-b9bb-4285-a8bd-f2966855e038-kube-api-access-qxlzr\") pod \"22a845fa-b9bb-4285-a8bd-f2966855e038\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") "
Jul 19 15:37:43.141175 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.141027    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a845fa-b9bb-4285-a8bd-f2966855e038-openshift-service-ca\") pod \"22a845fa-b9bb-4285-a8bd-f2966855e038\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") "
Jul 19 15:37:43.141175 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.141048    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls\") pod \"22a845fa-b9bb-4285-a8bd-f2966855e038\" (UID: \"22a845fa-b9bb-4285-a8bd-f2966855e038\") "
Jul 19 15:37:43.141381 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.141344    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22a845fa-b9bb-4285-a8bd-f2966855e038-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22a845fa-b9bb-4285-a8bd-f2966855e038" (UID: "22a845fa-b9bb-4285-a8bd-f2966855e038"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:37:43.143058 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.143038    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22a845fa-b9bb-4285-a8bd-f2966855e038" (UID: "22a845fa-b9bb-4285-a8bd-f2966855e038"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:37:43.143191 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.143171    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a845fa-b9bb-4285-a8bd-f2966855e038-kube-api-access-qxlzr" (OuterVolumeSpecName: "kube-api-access-qxlzr") pod "22a845fa-b9bb-4285-a8bd-f2966855e038" (UID: "22a845fa-b9bb-4285-a8bd-f2966855e038"). InnerVolumeSpecName "kube-api-access-qxlzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:43.242169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.242133    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxlzr\" (UniqueName: \"kubernetes.io/projected/22a845fa-b9bb-4285-a8bd-f2966855e038-kube-api-access-qxlzr\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:43.242169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.242161    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a845fa-b9bb-4285-a8bd-f2966855e038-openshift-service-ca\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:43.242169 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.242171    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a845fa-b9bb-4285-a8bd-f2966855e038-maas-api-tls\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:43.953308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.953221    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"
Jul 19 15:37:43.953308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.953258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw" event={"ID":"22a845fa-b9bb-4285-a8bd-f2966855e038","Type":"ContainerDied","Data":"fb2bde46e9c4aec84659da5b706411e3dc4306c1afceb0b7b81d46a9fd146e2d"}
Jul 19 15:37:43.953828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.953321    2577 scope.go:117] "RemoveContainer" containerID="2efe20c572737725cc819503e8c19219004e10d68dff75ea2e6d7572b9047f2a"
Jul 19 15:37:43.971384 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.971359    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"]
Jul 19 15:37:43.975837 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:43.975815    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ccd7ab-5fc897d8cc-l4qvw"]
Jul 19 15:37:45.539380 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:45.539352    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22a845fa-b9bb-4285-a8bd-f2966855e038" path="/var/lib/kubelet/pods/22a845fa-b9bb-4285-a8bd-f2966855e038/volumes"
Jul 19 15:37:47.344640 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:47.344610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:47.756322 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:47.756298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:53.768206 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:53.768085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:53.838114 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:53.838076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:54.024860 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:54.024786    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"]
Jul 19 15:37:54.025298 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:54.025018    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" containerID="cri-o://33d2e55034ed3e25ba778396959efc69acbde85672f90ea05ec5c0ccb94d4a8b" gracePeriod=30
Jul 19 15:37:54.045901 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:54.045864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:54.286276 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:54.286180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:37:54.742489 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:54.742462    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-595d697f79-vv4p9_5398e1d7-d450-4c63-a321-4eb9025dd4a6/authorino/0.log"
Jul 19 15:37:56.370903 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:56.370864    2577 patch_prober.go:28] interesting pod/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:37:56.371279 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:56.370918    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:37:59.287858 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.287801    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:37:59.378139 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378099    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-certs\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-podinfo\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378183    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs5pt\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-kube-api-access-fs5pt\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378205    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istiod-ca-cert\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-credential-socket\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378254    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-data\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378334 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-socket\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378341    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-envoy\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:59.378690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378392    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-token\") pod \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\" (UID: \"a5df19f1-366e-4ddc-af95-7cda19d14ccc\") "
Jul 19 15:37:59.378690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378610    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:59.378690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378671    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:59.378690 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378682    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:37:59.378995 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378767    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-certs\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.378995 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.378789    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-credential-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.380430 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.379293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:59.387895 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.387863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-kube-api-access-fs5pt" (OuterVolumeSpecName: "kube-api-access-fs5pt") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "kube-api-access-fs5pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:59.388001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.387936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:37:59.388001 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.387981    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:59.388203 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.388144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5df19f1-366e-4ddc-af95-7cda19d14ccc" (UID: "a5df19f1-366e-4ddc-af95-7cda19d14ccc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:59.479462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479408    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-podinfo\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.479462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479458    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs5pt\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-kube-api-access-fs5pt\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.479462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479469    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istiod-ca-cert\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.479462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479477    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-data\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.479462 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479485    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-workload-socket\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.479809 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479519    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-envoy\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.479809 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.479527    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5df19f1-366e-4ddc-af95-7cda19d14ccc-istio-token\") on node \"ip-10-0-142-137.ec2.internal\" DevicePath \"\""
Jul 19 15:37:59.883118 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.883034    2577 scope.go:117] "RemoveContainer" containerID="1b3890f5aa88328202d913412d3845a55284e73432e7f2058c359591c5b833cf"
Jul 19 15:37:59.891447 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:37:59.891423    2577 scope.go:117] "RemoveContainer" containerID="33d2e55034ed3e25ba778396959efc69acbde85672f90ea05ec5c0ccb94d4a8b"
Jul 19 15:38:00.032885 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.032843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" event={"ID":"a5df19f1-366e-4ddc-af95-7cda19d14ccc","Type":"ContainerDied","Data":"33d2e55034ed3e25ba778396959efc69acbde85672f90ea05ec5c0ccb94d4a8b"}
Jul 19 15:38:00.033065 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.032907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5" event={"ID":"a5df19f1-366e-4ddc-af95-7cda19d14ccc","Type":"ContainerDied","Data":"d37e750bd5fa21be8d7c511430def07e7abd4891b78ec23befc0f99230468e75"}
Jul 19 15:38:00.033065 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.032869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"
Jul 19 15:38:00.068154 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.068122    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"]
Jul 19 15:38:00.073963 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.073938    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ccd7ab-openshift-default-6df565fc59-f82l5"]
Jul 19 15:38:00.276105 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.276077    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-54k9g_dad499b2-7418-45cf-b851-5ceef13e66d8/manager/0.log"
Jul 19 15:38:00.409577 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.409543    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wjrft_b320f416-e8bb-45d2-a997-ca035bdcb944/manager/0.log"
Jul 19 15:38:00.547756 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.547676    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74d45bc5b5-2kh6h_2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58/manager/0.log"
Jul 19 15:38:00.919594 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.919520    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-6n4sq_fb14947d-0be7-4f6a-9ee6-df15c29672ca/copy-manifests/0.log"
Jul 19 15:38:00.938716 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:00.938690    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-6n4sq_fb14947d-0be7-4f6a-9ee6-df15c29672ca/manager/0.log"
Jul 19 15:38:01.539467 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:01.539438    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" path="/var/lib/kubelet/pods/a5df19f1-366e-4ddc-af95-7cda19d14ccc/volumes"
Jul 19 15:38:01.592700 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:01.592653    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-857956d86b-6qlhb_fadfe2bc-8556-4b67-ae13-1340d4721e3d/maas-api/0.log"
Jul 19 15:38:01.713647 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:01.713618    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29741235-x9dhc_84bfcad4-da42-4127-992d-d2f0579fbe06/cleanup/0.log"
Jul 19 15:38:01.958702 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:01.958600    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-82qjl_8cceab3a-9368-4286-8cca-e2c89e8ee996/postgres/0.log"
Jul 19 15:38:03.272733 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:03.272665    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-595d697f79-vv4p9_5398e1d7-d450-4c63-a321-4eb9025dd4a6/authorino/0.log"
Jul 19 15:38:03.391466 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:03.391437    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-m5dr6_f32c0840-2442-446e-a763-7b97b16bd6d9/manager/0.log"
Jul 19 15:38:03.507162 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:03.507133    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pknc5_4342cb54-d9a9-44e8-b7d6-f1c897fe93a8/manager/0.log"
Jul 19 15:38:03.744555 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:03.744527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vvhsr_6df328cc-31c0-483d-b4e9-5d30aaa3a356/registry-server/0.log"
Jul 19 15:38:04.115101 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:04.115021    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bfc9t_dab0e4ce-dd73-479f-b438-9d0cd0b3e950/manager/0.log"
Jul 19 15:38:04.493626 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:04.493598    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lpbm5_cf056216-dc88-48c4-a21e-8e959362da24/istio-proxy/0.log"
Jul 19 15:38:04.747404 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:04.747325    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-4kpsb_a4e16d34-9332-4118-9f73-5a896e0cdda5/kube-auth-proxy/0.log"
Jul 19 15:38:05.002693 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:05.002621    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/1.log"
Jul 19 15:38:05.134773 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:05.134744    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-56fl8_e9e1e882-27b2-45b6-8da0-868bdcc393bd/payload-pre-processing/0.log"
Jul 19 15:38:05.869785 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:05.869755    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-v4gpb_6dd4e8c5-f078-4125-890c-2e1b4f5a529f/main/0.log"
Jul 19 15:38:06.097673 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:06.097639    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fndkft_b6f151b9-d71d-4739-88de-b61a0e96d3e3/main/0.log"
Jul 19 15:38:12.344783 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:12.344747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:38:12.411239 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:12.411213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log" limit=1024
Jul 19 15:38:15.269848 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:15.269814    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kpllx_bd683c25-0c79-4e63-b8ba-6d1b14e2463f/global-pull-secret-syncer/0.log"
Jul 19 15:38:15.368660 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:15.368631    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2vssd_24aa3357-ff95-449c-b1b5-fda0fac5df4a/konnectivity-agent/0.log"
Jul 19 15:38:15.513131 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:15.513089    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-137.ec2.internal_6e72960bc383aaed664cf3e1030c9130/haproxy/0.log"
Jul 19 15:38:21.140043 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:21.139994    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-595d697f79-vv4p9_5398e1d7-d450-4c63-a321-4eb9025dd4a6/authorino/0.log"
Jul 19 15:38:21.168852 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:21.168823    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-m5dr6_f32c0840-2442-446e-a763-7b97b16bd6d9/manager/0.log"
Jul 19 15:38:21.193113 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:21.193093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pknc5_4342cb54-d9a9-44e8-b7d6-f1c897fe93a8/manager/0.log"
Jul 19 15:38:21.271156 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:21.271132    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vvhsr_6df328cc-31c0-483d-b4e9-5d30aaa3a356/registry-server/0.log"
Jul 19 15:38:21.632323 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:21.632287    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bfc9t_dab0e4ce-dd73-479f-b438-9d0cd0b3e950/manager/0.log"
Jul 19 15:38:23.722416 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:23.722391    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jvqhz_0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd/node-exporter/0.log"
Jul 19 15:38:23.742274 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:23.742252    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jvqhz_0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd/kube-rbac-proxy/0.log"
Jul 19 15:38:23.764549 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:23.764533    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jvqhz_0e13dd8a-9f3d-4479-a4cb-0e2c4c28f6bd/init-textfile/0.log"
Jul 19 15:38:24.239123 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:24.239082    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c745b67b9-gfj5n_81121ffb-4312-437a-8eab-ac80374e7c8a/telemeter-client/0.log"
Jul 19 15:38:24.265342 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:24.265312    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c745b67b9-gfj5n_81121ffb-4312-437a-8eab-ac80374e7c8a/reload/0.log"
Jul 19 15:38:24.288828 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:24.288806    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c745b67b9-gfj5n_81121ffb-4312-437a-8eab-ac80374e7c8a/kube-rbac-proxy/0.log"
Jul 19 15:38:25.835200 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835168    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"]
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835566    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" containerName="maas-api"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835579    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" containerName="maas-api"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835593    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a845fa-b9bb-4285-a8bd-f2966855e038" containerName="maas-api"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835598    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a845fa-b9bb-4285-a8bd-f2966855e038" containerName="maas-api"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835611    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835620    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835636    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy"
Jul 19 15:38:25.835670 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835641    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy"
Jul 19 15:38:25.835921 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835699    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a845fa-b9bb-4285-a8bd-f2966855e038" containerName="maas-api"
Jul 19 15:38:25.835921 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835707    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy"
Jul 19 15:38:25.835921 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835713    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5df19f1-366e-4ddc-af95-7cda19d14ccc" containerName="istio-proxy"
Jul 19 15:38:25.835921 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835718    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c55a53e0-83f1-42fa-bf4c-7e3f19937890" containerName="istio-proxy"
Jul 19 15:38:25.835921 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.835725    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2888aff3-e6d1-4f87-a85e-5a9384e3bdd8" containerName="maas-api"
Jul 19 15:38:25.838565 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.838547    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.840797 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.840759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bxflm\"/\"default-dockercfg-7krnv\""
Jul 19 15:38:25.840950 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.840764    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bxflm\"/\"openshift-service-ca.crt\""
Jul 19 15:38:25.841899 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.841879    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bxflm\"/\"kube-root-ca.crt\""
Jul 19 15:38:25.847325 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.847299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"]
Jul 19 15:38:25.894125 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.894087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-lib-modules\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.894245 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.894184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-sys\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.894245 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.894236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-proc\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.894323 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.894254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-podres\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.894323 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.894275    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grsj8\" (UniqueName: \"kubernetes.io/projected/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-kube-api-access-grsj8\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995619 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-sys\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995727 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-proc\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995727 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-podres\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995727 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grsj8\" (UniqueName: \"kubernetes.io/projected/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-kube-api-access-grsj8\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995727 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-sys\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995727 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-lib-modules\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-podres\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-proc\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:25.995896 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:25.995838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-lib-modules\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:26.003590 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:26.003569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grsj8\" (UniqueName: \"kubernetes.io/projected/9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2-kube-api-access-grsj8\") pod \"perf-node-gather-daemonset-7rcn8\" (UID: \"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:26.148993 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:26.148924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:26.283276 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:26.283079    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"]
Jul 19 15:38:26.285467 ip-10-0-142-137 kubenswrapper[2577]: W0719 15:38:26.285433    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9d3d5cee_bbd1_4a68_9ef3_954e20fdd1f2.slice/crio-1a4ba1055fb91027d2093cd5fc692bb28f38f7409df7cacc5f6a9bdb499a9996 WatchSource:0}: Error finding container 1a4ba1055fb91027d2093cd5fc692bb28f38f7409df7cacc5f6a9bdb499a9996: Status 404 returned error can't find the container with id 1a4ba1055fb91027d2093cd5fc692bb28f38f7409df7cacc5f6a9bdb499a9996
Jul 19 15:38:27.159849 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:27.159812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8" event={"ID":"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2","Type":"ContainerStarted","Data":"93495d02fbe9d27f9146c15d4046d78530890a2ba1fe4565046923c8861c3906"}
Jul 19 15:38:27.160231 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:27.159859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8" event={"ID":"9d3d5cee-bbd1-4a68-9ef3-954e20fdd1f2","Type":"ContainerStarted","Data":"1a4ba1055fb91027d2093cd5fc692bb28f38f7409df7cacc5f6a9bdb499a9996"}
Jul 19 15:38:27.160231 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:27.159891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:27.177843 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:27.177799    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8" podStartSLOduration=2.177784883 podStartE2EDuration="2.177784883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 15:38:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:38:27.175704077 +0000 UTC m=+3088.200713151" watchObservedRunningTime="2026-07-19 15:38:27.177784883 +0000 UTC m=+3088.202793957"
Jul 19 15:38:28.475832 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:28.475733    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8rllc_7af39c6e-cb98-4b68-a6de-074e52edfd22/dns/0.log"
Jul 19 15:38:28.498819 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:28.498789    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8rllc_7af39c6e-cb98-4b68-a6de-074e52edfd22/kube-rbac-proxy/0.log"
Jul 19 15:38:28.673188 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:28.673163    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wxdbs_fa88057d-da49-49e6-ac70-a17af00c1060/dns-node-resolver/0.log"
Jul 19 15:38:29.252261 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:29.252230    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7n5n2_9af30ba0-f27d-45f2-91d7-bedb8f428ad7/node-ca/0.log"
Jul 19 15:38:30.332757 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:30.332727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lpbm5_cf056216-dc88-48c4-a21e-8e959362da24/istio-proxy/0.log"
Jul 19 15:38:30.485276 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:30.485241    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-4kpsb_a4e16d34-9332-4118-9f73-5a896e0cdda5/kube-auth-proxy/0.log"
Jul 19 15:38:30.544518 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:30.544476    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/0.log"
Jul 19 15:38:30.595567 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:30.595464    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/1.log"
Jul 19 15:38:30.633060 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:30.633034    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-56fl8_e9e1e882-27b2-45b6-8da0-868bdcc393bd/payload-pre-processing/0.log"
Jul 19 15:38:31.433909 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:31.433882    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d5lrq_fd583199-d177-4296-a397-e3f9e45ac406/serve-healthcheck-canary/0.log"
Jul 19 15:38:32.072906 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:32.072871    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5xjc_77208df7-adea-4cbd-88fb-9444a27a1afa/kube-rbac-proxy/0.log"
Jul 19 15:38:32.094798 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:32.094774    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5xjc_77208df7-adea-4cbd-88fb-9444a27a1afa/exporter/0.log"
Jul 19 15:38:32.118398 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:32.118377    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5xjc_77208df7-adea-4cbd-88fb-9444a27a1afa/extractor/0.log"
Jul 19 15:38:33.174847 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:33.174819    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-7rcn8"
Jul 19 15:38:34.340619 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:34.340593    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-54k9g_dad499b2-7418-45cf-b851-5ceef13e66d8/manager/0.log"
Jul 19 15:38:34.685895 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:34.685804    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wjrft_b320f416-e8bb-45d2-a997-ca035bdcb944/manager/0.log"
Jul 19 15:38:34.881704 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:34.881672    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74d45bc5b5-2kh6h_2e2eb6f1-4357-4b7a-b91f-2a906bcc8f58/manager/0.log"
Jul 19 15:38:35.133993 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:35.133956    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-6n4sq_fb14947d-0be7-4f6a-9ee6-df15c29672ca/manager/0.log"
Jul 19 15:38:35.149128 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:35.149106    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-6n4sq_fb14947d-0be7-4f6a-9ee6-df15c29672ca/copy-manifests/0.log"
Jul 19 15:38:36.463195 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:36.463164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lpbm5_cf056216-dc88-48c4-a21e-8e959362da24/istio-proxy/0.log"
Jul 19 15:38:36.587857 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:36.587828    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-4kpsb_a4e16d34-9332-4118-9f73-5a896e0cdda5/kube-auth-proxy/0.log"
Jul 19 15:38:36.622747 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:36.622723    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/0.log"
Jul 19 15:38:36.629514 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:36.629475    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sfvmm_3717f16a-c5b0-46bf-9468-f7f7128fe3ec/openshift-lws-operator/0.log"
Jul 19 15:38:36.672967 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:36.672947    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/1.log"
Jul 19 15:38:36.699413 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:36.699373    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-56fl8_e9e1e882-27b2-45b6-8da0-868bdcc393bd/payload-pre-processing/0.log"
Jul 19 15:38:38.003208 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:38.003164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lpbm5_cf056216-dc88-48c4-a21e-8e959362da24/istio-proxy/0.log"
Jul 19 15:38:38.150266 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:38.150230    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-4kpsb_a4e16d34-9332-4118-9f73-5a896e0cdda5/kube-auth-proxy/0.log"
Jul 19 15:38:38.187308 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:38.187275    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/0.log"
Jul 19 15:38:38.241654 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:38.241628    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-znlwk_11d01314-3d84-44cc-b5c7-9cab54e14dea/istio-proxy/1.log"
Jul 19 15:38:38.275198 ip-10-0-142-137 kubenswrapper[2577]: I0719 15:38:38.275121    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-56fl8_e9e1e882-27b2-45b6-8da0-868bdcc393bd/payload-pre-processing/0.log"